DOMAIN: Electronics and Telecommunication
CONTEXT: A communications equipment manufacturing company has a product which is responsible for emitting informative signals. Company wants to build a machine learning model which can help the company to predict the equipment’s signal quality using various parameters
DATA DESCRIPTION: The data set contains information on various signal tests performed:
PROJECT OBJECTIVE: The need is to build a regressor which can use these parameters to determine the signal strength or quality [as number]
Steps and tasks:
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from statsmodels.stats.multicomp import pairwise_tukeyhsd
from scipy.stats import ttest_ind, levene, shapiro
from sklearn.preprocessing import StandardScaler
import tensorflow
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras import regularizers, optimizers
# Load csv file into master dataframe
df_master = pd.read_csv("Part- 1 - Signal.csv")
# Separating dependant and independent variables
X_train = df_master.iloc[:, :-1]
y_train = df_master.iloc[:, -1]
# Normalising the independent variables
scaler = StandardScaler()
X_train = pd.DataFrame(scaler.fit_transform(X_train), columns=df_master.iloc[:, :-1].columns)
# setting up validation, testing and training sets
X_train, X_test, y_train, y_test = train_test_split(X_train, y_train, test_size=0.5, random_state=42, stratify=y_train)
X_val, X_test, y_val, y_test = train_test_split(X_test, y_test, test_size=0.5, random_state=69, stratify=y_test)
# View basic stats of dataframe
df_master.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 1599 entries, 0 to 1598 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Parameter 1 1599 non-null float64 1 Parameter 2 1599 non-null float64 2 Parameter 3 1599 non-null float64 3 Parameter 4 1599 non-null float64 4 Parameter 5 1599 non-null float64 5 Parameter 6 1599 non-null float64 6 Parameter 7 1599 non-null float64 7 Parameter 8 1599 non-null float64 8 Parameter 9 1599 non-null float64 9 Parameter 10 1599 non-null float64 10 Parameter 11 1599 non-null float64 11 Signal_Strength 1599 non-null int64 dtypes: float64(11), int64(1) memory usage: 150.0 KB
# View detailed stats of the dataframe
df_master.describe()
| Parameter 1 | Parameter 2 | Parameter 3 | Parameter 4 | Parameter 5 | Parameter 6 | Parameter 7 | Parameter 8 | Parameter 9 | Parameter 10 | Parameter 11 | Signal_Strength | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| count | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 |
| mean | 8.319637 | 0.527821 | 0.270976 | 2.538806 | 0.087467 | 15.874922 | 46.467792 | 0.996747 | 3.311113 | 0.658149 | 10.422983 | 5.636023 |
| std | 1.741096 | 0.179060 | 0.194801 | 1.409928 | 0.047065 | 10.460157 | 32.895324 | 0.001887 | 0.154386 | 0.169507 | 1.065668 | 0.807569 |
| min | 4.600000 | 0.120000 | 0.000000 | 0.900000 | 0.012000 | 1.000000 | 6.000000 | 0.990070 | 2.740000 | 0.330000 | 8.400000 | 3.000000 |
| 25% | 7.100000 | 0.390000 | 0.090000 | 1.900000 | 0.070000 | 7.000000 | 22.000000 | 0.995600 | 3.210000 | 0.550000 | 9.500000 | 5.000000 |
| 50% | 7.900000 | 0.520000 | 0.260000 | 2.200000 | 0.079000 | 14.000000 | 38.000000 | 0.996750 | 3.310000 | 0.620000 | 10.200000 | 6.000000 |
| 75% | 9.200000 | 0.640000 | 0.420000 | 2.600000 | 0.090000 | 21.000000 | 62.000000 | 0.997835 | 3.400000 | 0.730000 | 11.100000 | 6.000000 |
| max | 15.900000 | 1.580000 | 1.000000 | 15.500000 | 0.611000 | 72.000000 | 289.000000 | 1.003690 | 4.010000 | 2.000000 | 14.900000 | 8.000000 |
print("Train")
print(np.unique(y_train, return_counts=True))
print("Test")
print(np.unique(y_test, return_counts=True))
print("Val")
print(np.unique(y_val, return_counts=True))
Train (array([3, 4, 5, 6, 7, 8], dtype=int64), array([ 5, 27, 340, 319, 99, 9], dtype=int64)) Test (array([3, 4, 5, 6, 7, 8], dtype=int64), array([ 2, 13, 171, 160, 50, 4], dtype=int64)) Val (array([3, 4, 5, 6, 7, 8], dtype=int64), array([ 3, 13, 170, 159, 50, 5], dtype=int64))
# Performing Tukey's hsd test on different groups, based on Class variable, for all independent variables
# Running a for loop to extract each attribute name individually
for col in df_master.columns[:-1]:
# Printing newline for cosmetic purposes
print("\n", col, "\n")
# Display Tukey hsd test results
print(pairwise_tukeyhsd(df_master[col], df_master["Signal_Strength"], alpha=0.05))
Parameter 1
Multiple Comparison of Means - Tukey HSD, FWER=0.05
===================================================
group1 group2 meandiff p-adj lower upper reject
---------------------------------------------------
3 4 -0.5808 0.9 -2.2795 1.118 False
3 5 -0.1927 0.9 -1.7622 1.3767 False
3 6 -0.0128 0.9 -1.5831 1.5574 False
3 7 0.5124 0.9 -1.0844 2.1091 False
3 8 0.2067 0.9 -1.7366 2.1499 False
4 5 0.388 0.5991 -0.3146 1.0906 False
4 6 0.5679 0.1942 -0.1364 1.2723 False
4 7 1.0931 0.001 0.3315 1.8547 True
4 8 0.7874 0.5438 -0.5567 2.1316 False
5 6 0.1799 0.4099 -0.0915 0.4514 False
5 7 0.7051 0.001 0.3081 1.1021 True
5 8 0.3994 0.9 -0.7772 1.576 False
6 7 0.5252 0.0026 0.1251 0.9252 True
6 8 0.2195 0.9 -0.9581 1.3971 False
7 8 -0.3057 0.9 -1.5184 0.907 False
---------------------------------------------------
Parameter 2
Multiple Comparison of Means - Tukey HSD, FWER=0.05
====================================================
group1 group2 meandiff p-adj lower upper reject
----------------------------------------------------
3 4 -0.1905 0.0102 -0.3522 -0.0289 True
3 5 -0.3075 0.001 -0.4568 -0.1581 True
3 6 -0.387 0.001 -0.5364 -0.2376 True
3 7 -0.4806 0.001 -0.6325 -0.3286 True
3 8 -0.4612 0.001 -0.6461 -0.2763 True
4 5 -0.1169 0.001 -0.1838 -0.0501 True
4 6 -0.1965 0.001 -0.2635 -0.1295 True
4 7 -0.29 0.001 -0.3625 -0.2176 True
4 8 -0.2706 0.001 -0.3985 -0.1427 True
5 6 -0.0796 0.001 -0.1054 -0.0537 True
5 7 -0.1731 0.001 -0.2109 -0.1353 True
5 8 -0.1537 0.0013 -0.2657 -0.0418 True
6 7 -0.0936 0.001 -0.1316 -0.0555 True
6 8 -0.0742 0.4117 -0.1862 0.0379 False
7 8 0.0194 0.9 -0.096 0.1348 False
----------------------------------------------------
Parameter 3
Multiple Comparison of Means - Tukey HSD, FWER=0.05
===================================================
group1 group2 meandiff p-adj lower upper reject
---------------------------------------------------
3 4 0.0032 0.9 -0.1831 0.1894 False
3 5 0.0727 0.8128 -0.0994 0.2448 False
3 6 0.1028 0.525 -0.0693 0.275 False
3 7 0.2042 0.0115 0.0291 0.3792 True
3 8 0.2201 0.0382 0.007 0.4332 True
4 5 0.0695 0.104 -0.0075 0.1466 False
4 6 0.0997 0.0033 0.0224 0.1769 True
4 7 0.201 0.001 0.1175 0.2845 True
4 8 0.217 0.001 0.0696 0.3643 True
5 6 0.0301 0.0452 0.0004 0.0599 True
5 7 0.1315 0.001 0.088 0.175 True
5 8 0.1474 0.0144 0.0184 0.2764 True
6 7 0.1014 0.001 0.0575 0.1452 True
6 8 0.1173 0.0997 -0.0118 0.2464 False
7 8 0.0159 0.9 -0.117 0.1489 False
---------------------------------------------------
Parameter 4
Multiple Comparison of Means - Tukey HSD, FWER=0.05
===================================================
group1 group2 meandiff p-adj lower upper reject
---------------------------------------------------
3 4 0.0593 0.9 -1.3275 1.4462 False
3 5 -0.1061 0.9 -1.3874 1.1752 False
3 6 -0.1578 0.9 -1.4397 1.1241 False
3 7 0.0856 0.9 -1.218 1.3892 False
3 8 -0.0572 0.9 -1.6437 1.5292 False
4 5 -0.1655 0.9 -0.7391 0.4081 False
4 6 -0.2171 0.8865 -0.7922 0.3579 False
4 7 0.0263 0.9 -0.5955 0.648 False
4 8 -0.1166 0.9 -1.2139 0.9808 False
5 6 -0.0517 0.9 -0.2733 0.17 False
5 7 0.1917 0.5343 -0.1324 0.5159 False
5 8 0.0489 0.9 -0.9116 1.0095 False
6 7 0.2434 0.2741 -0.0832 0.57 False
6 8 0.1006 0.9 -0.8608 1.062 False
7 8 -0.1428 0.9 -1.1329 0.8472 False
---------------------------------------------------
Parameter 5
Multiple Comparison of Means - Tukey HSD, FWER=0.05
====================================================
group1 group2 meandiff p-adj lower upper reject
----------------------------------------------------
3 4 -0.0318 0.357 -0.0778 0.0141 False
3 5 -0.0298 0.3424 -0.0722 0.0127 False
3 6 -0.0375 0.1183 -0.08 0.0049 False
3 7 -0.0459 0.0295 -0.0891 -0.0027 True
3 8 -0.0541 0.0396 -0.1066 -0.0015 True
4 5 0.0021 0.9 -0.0169 0.0211 False
4 6 -0.0057 0.9 -0.0248 0.0133 False
4 7 -0.0141 0.3717 -0.0347 0.0065 False
4 8 -0.0222 0.5012 -0.0586 0.0141 False
5 6 -0.0078 0.0305 -0.0151 -0.0004 True
5 7 -0.0161 0.001 -0.0269 -0.0054 True
5 8 -0.0243 0.2483 -0.0561 0.0075 False
6 7 -0.0084 0.235 -0.0192 0.0025 False
6 8 -0.0165 0.6546 -0.0484 0.0153 False
7 8 -0.0081 0.9 -0.0409 0.0247 False
----------------------------------------------------
Parameter 6
Multiple Comparison of Means - Tukey HSD, FWER=0.05
=====================================================
group1 group2 meandiff p-adj lower upper reject
-----------------------------------------------------
3 4 1.2642 0.9 -8.9655 11.4938 False
3 5 5.9838 0.4634 -3.4675 15.4352 False
3 6 4.7116 0.6879 -4.7443 14.1675 False
3 7 3.0452 0.9 -6.5703 12.6608 False
3 8 2.2778 0.9 -9.4245 13.98 False
4 5 4.7197 0.0186 0.4885 8.9509 True
4 6 3.4474 0.1867 -0.794 7.6889 False
4 7 1.7811 0.8689 -2.8052 6.3674 False
4 8 1.0136 0.9 -7.0807 9.108 False
5 6 -1.2722 0.2288 -2.9071 0.3626 False
5 7 -2.9386 0.0062 -5.3296 -0.5477 True
5 8 -3.7061 0.6471 -10.7913 3.3792 False
6 7 -1.6664 0.3587 -4.0755 0.7427 False
6 8 -2.4338 0.9 -9.5252 4.6576 False
7 8 -0.7674 0.9 -8.0703 6.5354 False
-----------------------------------------------------
Parameter 7
Multiple Comparison of Means - Tukey HSD, FWER=0.05
======================================================
group1 group2 meandiff p-adj lower upper reject
------------------------------------------------------
3 4 11.3453 0.9 -19.8412 42.5318 False
3 5 31.614 0.0219 2.8002 60.4277 True
3 6 15.9699 0.5962 -12.8579 44.7977 False
3 7 10.1201 0.9 -19.1943 39.4345 False
3 8 8.5444 0.9 -27.1316 44.2205 False
4 5 20.2687 0.001 7.3692 33.1681 True
4 6 4.6246 0.9 -8.3062 17.5554 False
4 7 -1.2252 0.9 -15.2072 12.7568 False
4 8 -2.8008 0.9 -27.4776 21.876 False
5 6 -15.644 0.001 -20.628 -10.6601 True
5 7 -21.4938 0.001 -28.783 -14.2047 True
5 8 -23.0695 0.0284 -44.6699 -1.4691 True
6 7 -5.8498 0.2059 -13.1943 1.4947 False
6 8 -7.4255 0.9 -29.0446 14.1937 False
7 8 -1.5757 0.9 -23.8395 20.6882 False
------------------------------------------------------
Parameter 8
Multiple Comparison of Means - Tukey HSD, FWER=0.05
====================================================
group1 group2 meandiff p-adj lower upper reject
----------------------------------------------------
3 4 -0.0009 0.6753 -0.0027 0.0009 False
3 5 -0.0004 0.9 -0.002 0.0013 False
3 6 -0.0008 0.6782 -0.0025 0.0008 False
3 7 -0.0014 0.2088 -0.0031 0.0004 False
3 8 -0.0023 0.0254 -0.0043 -0.0002 True
4 5 0.0006 0.2748 -0.0002 0.0013 False
4 6 0.0001 0.9 -0.0007 0.0008 False
4 7 -0.0004 0.6249 -0.0013 0.0004 False
4 8 -0.0013 0.0898 -0.0028 0.0001 False
5 6 -0.0005 0.001 -0.0008 -0.0002 True
5 7 -0.001 0.001 -0.0014 -0.0006 True
5 8 -0.0019 0.001 -0.0032 -0.0006 True
6 7 -0.0005 0.0091 -0.0009 -0.0001 True
6 8 -0.0014 0.0194 -0.0027 -0.0001 True
7 8 -0.0009 0.3686 -0.0022 0.0004 False
----------------------------------------------------
Parameter 9
Multiple Comparison of Means - Tukey HSD, FWER=0.05
====================================================
group1 group2 meandiff p-adj lower upper reject
----------------------------------------------------
3 4 -0.0165 0.9 -0.1676 0.1346 False
3 5 -0.0931 0.4029 -0.2326 0.0465 False
3 6 -0.0799 0.566 -0.2196 0.0597 False
3 7 -0.1072 0.2599 -0.2493 0.0348 False
3 8 -0.1308 0.2578 -0.3036 0.0421 False
4 5 -0.0766 0.0064 -0.1391 -0.0141 True
4 6 -0.0634 0.0452 -0.1261 -0.0008 True
4 7 -0.0908 0.0019 -0.1585 -0.023 True
4 8 -0.1143 0.0706 -0.2338 0.0053 False
5 6 0.0131 0.6134 -0.011 0.0373 False
5 7 -0.0142 0.8464 -0.0495 0.0211 False
5 8 -0.0377 0.9 -0.1424 0.0669 False
6 7 -0.0273 0.2424 -0.0629 0.0083 False
6 8 -0.0508 0.7089 -0.1556 0.0539 False
7 8 -0.0235 0.9 -0.1314 0.0843 False
----------------------------------------------------
Parameter 10
Multiple Comparison of Means - Tukey HSD, FWER=0.05
===================================================
group1 group2 meandiff p-adj lower upper reject
---------------------------------------------------
3 4 0.0264 0.9 -0.135 0.1879 False
3 5 0.051 0.9 -0.0982 0.2001 False
3 6 0.1053 0.3349 -0.0439 0.2546 False
3 7 0.1713 0.0165 0.0195 0.323 True
3 8 0.1978 0.0277 0.0131 0.3825 True
4 5 0.0246 0.9 -0.0422 0.0913 False
4 6 0.0789 0.0102 0.012 0.1459 True
4 7 0.1448 0.001 0.0725 0.2172 True
4 8 0.1714 0.0019 0.0436 0.2991 True
5 6 0.0544 0.001 0.0286 0.0802 True
5 7 0.1203 0.001 0.0826 0.158 True
5 8 0.1468 0.0026 0.035 0.2586 True
6 7 0.0659 0.001 0.0279 0.104 True
6 8 0.0924 0.1726 -0.0195 0.2044 False
7 8 0.0265 0.9 -0.0887 0.1418 False
---------------------------------------------------
Parameter 11
Multiple Comparison of Means - Tukey HSD, FWER=0.05
===================================================
group1 group2 meandiff p-adj lower upper reject
---------------------------------------------------
3 4 0.3101 0.9 -0.589 1.2092 False
3 5 -0.0553 0.9 -0.886 0.7754 False
3 6 0.6745 0.1881 -0.1566 1.5056 False
3 7 1.5109 0.001 0.6658 2.356 True
3 8 2.1394 0.001 1.1109 3.168 True
4 5 -0.3654 0.0575 -0.7373 0.0065 False
4 6 0.3644 0.0598 -0.0084 0.7372 False
4 7 1.2008 0.001 0.7977 1.6039 True
4 8 1.8294 0.001 1.1179 2.5408 True
5 6 0.7298 0.001 0.5861 0.8735 True
5 7 1.5662 0.001 1.3561 1.7764 True
5 8 2.1947 0.001 1.572 2.8175 True
6 7 0.8364 0.001 0.6247 1.0481 True
6 8 1.4649 0.001 0.8416 2.0882 True
7 8 0.6285 0.059 -0.0133 1.2704 False
---------------------------------------------------
# Check the difference in standard deviation between validation and test set against train set
print("Validation\n")
# Print the difference in std
print(X_train.std() - X_val.std())
print("\nTest\n")
# Print the difference in std
print(X_train.std() - X_test.std())
Validation Parameter 1 -0.013613 Parameter 2 0.007935 Parameter 3 -0.019101 Parameter 4 -0.184162 Parameter 5 0.083585 Parameter 6 0.032207 Parameter 7 -0.008781 Parameter 8 -0.006019 Parameter 9 0.018046 Parameter 10 0.301464 Parameter 11 0.005441 dtype: float64 Test Parameter 1 -0.012933 Parameter 2 0.024601 Parameter 3 0.008648 Parameter 4 -0.141007 Parameter 5 0.303672 Parameter 6 -0.016860 Parameter 7 -0.016655 Parameter 8 -0.023491 Parameter 9 -0.034784 Parameter 10 0.213939 Parameter 11 0.001439 dtype: float64
# Loop through all columns
for col in df_master.columns[:-1]:
# Print column name
print(col)
# Run a Welch test if Parameter 4, 5 or 10 else run t-test
if col in ["Parameter 4, Parameter 5, Parameter 10"]:
print("Validation set: ", ttest_ind(X_train[col], X_val[col], equal_var = False))
print("Test set: ", ttest_ind(X_train[col], X_test[col], equal_var = False))
else:
print("Validation set: ", ttest_ind(X_train[col], X_val[col]))
print("Test set: ", ttest_ind(X_train[col], X_test[col]))
Parameter 1 Validation set: Ttest_indResult(statistic=0.3706527253849732, pvalue=0.7109617864922428) Test set: Ttest_indResult(statistic=1.1906523572200014, pvalue=0.234026151364803) Parameter 2 Validation set: Ttest_indResult(statistic=0.7210102948796809, pvalue=0.47104397151023725) Test set: Ttest_indResult(statistic=0.9357710978552741, pvalue=0.34957980064120686) Parameter 3 Validation set: Ttest_indResult(statistic=0.33128481929614656, pvalue=0.7404873440335276) Test set: Ttest_indResult(statistic=1.0189233262210697, pvalue=0.308445260944195) Parameter 4 Validation set: Ttest_indResult(statistic=-0.8319281132589157, pvalue=0.4056154547111238) Test set: Ttest_indResult(statistic=0.0708404317518916, pvalue=0.9435365860531325) Parameter 5 Validation set: Ttest_indResult(statistic=0.6255255444271164, pvalue=0.5317454269898378) Test set: Ttest_indResult(statistic=1.3813785537459684, pvalue=0.167420390419297) Parameter 6 Validation set: Ttest_indResult(statistic=0.038996127353225794, pvalue=0.9688999788605794) Test set: Ttest_indResult(statistic=-0.12390502188286101, pvalue=0.9014112790366531) Parameter 7 Validation set: Ttest_indResult(statistic=0.41703419075948966, pvalue=0.676728177850991) Test set: Ttest_indResult(statistic=0.18943373450077344, pvalue=0.8497850023761182) Parameter 8 Validation set: Ttest_indResult(statistic=0.3676294674063405, pvalue=0.7132145302037807) Test set: Ttest_indResult(statistic=1.1653298874666678, pvalue=0.24411764993995463) Parameter 9 Validation set: Ttest_indResult(statistic=-0.7725654605870329, pvalue=0.4399321052447205) Test set: Ttest_indResult(statistic=-1.3678663676277107, pvalue=0.1716106443264137) Parameter 10 Validation set: Ttest_indResult(statistic=2.401139848459023, pvalue=0.016495876660595318) Test set: Ttest_indResult(statistic=1.597553005580714, pvalue=0.11040642224182128) Parameter 11 Validation set: Ttest_indResult(statistic=-0.26081809652882565, pvalue=0.7942776510916161) Test set: Ttest_indResult(statistic=0.3640082619612094, pvalue=0.7159161253881069)
Hint: Use your best analytical approach. Even you can mix match columns to create new columns which can be used for better analysis. Create your own features if required. Be highly experimental and analytical here to find relevant hidden patterns.
# Create the subplots
f, axes = plt.subplots(11, 1, figsize=(15,80))
cnt=0
# Automate each feature histogram
for col in df_master.columns[:-1]:
sns.histplot(df_master[col], ax=axes[cnt])
axes[cnt].set_title(col)
cnt+=1
f.tight_layout()
plt.show()
# Multivariate Analysis using bivariate pairplot
sns.pairplot(df_master, hue="Signal_Strengtha
<seaborn.axisgrid.PairGrid at 0x16d50b2bca0>
# Separating rows by either train, val or test set group
grp = []
for idx in range(len(df_master)):
if idx in X_train.index:
grp.append('Val')
elif idx in X_test.index:
grp.append('Test')
else:
grp.append('Train')
df_master["Group"] = grp
# Multivariate Analysis using bivariate pairplot
sns.pairplot(df_master.drop("Signal_Strength", axis=1), hue="Group")
<seaborn.axisgrid.PairGrid at 0x16d5c7cf940>
# Bivariate analysis
plt.figure(figsize = (30, 30))
sns.scatterplot(data=df_master, x="Parameter 2", y="Parameter 3", hue="Signal_Strength", size="Group", sizes=[50, 150, 200]
, size_order=["Train", "Val", "Test"])
plt.show()
Hint: Use best approach to refine and tune the data or the model. Be highly experimental here
# Setting up train, validation and test set target variable for Neural network classifier
y_train = tensorflow.keras.utils.to_categorical(y_train-3, num_classes=6)
y_val = tensorflow.keras.utils.to_categorical(y_val-3, num_classes=6)
y_test = tensorflow.keras.utils.to_categorical(y_test-3, num_classes=6)
def train_and_test_loop(iterations, lr, Lambda, verb=True):
## hyperparameters
iterations = iterations
learning_rate = lr
hidden_node1 = 26
hidden_node2 = 14
output_nodes = 6
model = Sequential()
model.add(Dense(hidden_node1, input_shape=(11,), activation='relu'))
model.add(Dense(hidden_node2, activation='relu'))
model.add(Dense(output_nodes, activation='softmax', kernel_regularizer=regularizers.l2(Lambda)))
sgd = optimizers.SGD(lr=learning_rate, decay=1e-6, momentum=0.9)
# Compile model
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
# Fit the model
model.fit(np.array(X_train), y_train, epochs=iterations, batch_size=1000, verbose= 1)
lr = 1e-2
Lambda = 1e-7
train_and_test_loop(20, lr, Lambda)
Epoch 1/20 1/1 [==============================] - 0s 0s/step - loss: 2.1247 - accuracy: 0.0526 Epoch 2/20 1/1 [==============================] - 0s 0s/step - loss: 2.1091 - accuracy: 0.0588 Epoch 3/20 1/1 [==============================] - 0s 8ms/step - loss: 2.0804 - accuracy: 0.0663 Epoch 4/20 1/1 [==============================] - 0s 0s/step - loss: 2.0414 - accuracy: 0.0826 Epoch 5/20 1/1 [==============================] - 0s 8ms/step - loss: 1.9947 - accuracy: 0.0976 Epoch 6/20 1/1 [==============================] - 0s 0s/step - loss: 1.9430 - accuracy: 0.1189 Epoch 7/20 1/1 [==============================] - 0s 0s/step - loss: 1.8893 - accuracy: 0.1402 Epoch 8/20 1/1 [==============================] - 0s 0s/step - loss: 1.8355 - accuracy: 0.1790 Epoch 9/20 1/1 [==============================] - 0s 0s/step - loss: 1.7835 - accuracy: 0.2190 Epoch 10/20 1/1 [==============================] - 0s 0s/step - loss: 1.7339 - accuracy: 0.2591 Epoch 11/20 1/1 [==============================] - 0s 0s/step - loss: 1.6871 - accuracy: 0.3079 Epoch 12/20 1/1 [==============================] - 0s 0s/step - loss: 1.6430 - accuracy: 0.3467 Epoch 13/20 1/1 [==============================] - 0s 8ms/step - loss: 1.6021 - accuracy: 0.3680 Epoch 14/20 1/1 [==============================] - 0s 0s/step - loss: 1.5643 - accuracy: 0.3892 Epoch 15/20 1/1 [==============================] - 0s 0s/step - loss: 1.5293 - accuracy: 0.4168 Epoch 16/20 1/1 [==============================] - 0s 0s/step - loss: 1.4971 - accuracy: 0.4355 Epoch 17/20 1/1 [==============================] - 0s 0s/step - loss: 1.4675 - accuracy: 0.4456 Epoch 18/20 1/1 [==============================] - 0s 0s/step - loss: 1.4403 - accuracy: 0.4593 Epoch 19/20 1/1 [==============================] - 0s 0s/step - loss: 1.4151 - accuracy: 0.4656 Epoch 20/20 1/1 [==============================] - 0s 0s/step - loss: 1.3919 - accuracy: 0.4656
lr = 0.01
Lambda = 0
train_and_test_loop(500, lr, Lambda)
Epoch 1/500 1/1 [==============================] - 0s 0s/step - loss: 1.9885 - accuracy: 0.0500 Epoch 2/500 1/1 [==============================] - 0s 0s/step - loss: 1.9708 - accuracy: 0.0500 Epoch 3/500 1/1 [==============================] - 0s 0s/step - loss: 1.9380 - accuracy: 0.0500 Epoch 4/500 1/1 [==============================] - 0s 0s/step - loss: 1.8930 - accuracy: 0.0500 Epoch 5/500 1/1 [==============================] - 0s 0s/step - loss: 1.8389 - accuracy: 0.0500 Epoch 6/500 1/1 [==============================] - 0s 0s/step - loss: 1.7784 - accuracy: 0.1000 Epoch 7/500 1/1 [==============================] - 0s 0s/step - loss: 1.7134 - accuracy: 0.1500 Epoch 8/500 1/1 [==============================] - 0s 0s/step - loss: 1.6464 - accuracy: 0.3500 Epoch 9/500 1/1 [==============================] - 0s 0s/step - loss: 1.5784 - accuracy: 0.4500 Epoch 10/500 1/1 [==============================] - 0s 8ms/step - loss: 1.5099 - accuracy: 0.5000 Epoch 11/500 1/1 [==============================] - 0s 0s/step - loss: 1.4412 - accuracy: 0.5000 Epoch 12/500 1/1 [==============================] - 0s 0s/step - loss: 1.3730 - accuracy: 0.6000 Epoch 13/500 1/1 [==============================] - 0s 8ms/step - loss: 1.3066 - accuracy: 0.6500 Epoch 14/500 1/1 [==============================] - 0s 0s/step - loss: 1.2421 - accuracy: 0.6500 Epoch 15/500 1/1 [==============================] - 0s 0s/step - loss: 1.1791 - accuracy: 0.6500 Epoch 16/500 1/1 [==============================] - 0s 0s/step - loss: 1.1183 - accuracy: 0.6500 Epoch 17/500 1/1 [==============================] - 0s 0s/step - loss: 1.0611 - accuracy: 0.7000 Epoch 18/500 1/1 [==============================] - 0s 0s/step - loss: 1.0089 - accuracy: 0.7000 Epoch 19/500 1/1 [==============================] - 0s 0s/step - loss: 0.9617 - accuracy: 0.6500 Epoch 20/500 1/1 [==============================] - 0s 0s/step - loss: 0.9204 - accuracy: 0.7000 Epoch 21/500 1/1 [==============================] - 0s 0s/step - loss: 0.8851 - accuracy: 0.7000 Epoch 22/500 1/1 [==============================] - 0s 0s/step - loss: 0.8552 - accuracy: 0.7000 Epoch 23/500 1/1 [==============================] - 0s 0s/step - loss: 0.8302 - accuracy: 0.7000 Epoch 24/500 1/1 [==============================] - 0s 0s/step - loss: 0.8097 - accuracy: 0.7000 Epoch 25/500 1/1 [==============================] - 0s 0s/step - loss: 0.7927 - accuracy: 0.7000 Epoch 26/500 1/1 [==============================] - 0s 8ms/step - loss: 0.7781 - accuracy: 0.7000 Epoch 27/500 1/1 [==============================] - 0s 0s/step - loss: 0.7653 - accuracy: 0.7000 Epoch 28/500 1/1 [==============================] - 0s 0s/step - loss: 0.7536 - accuracy: 0.7000 Epoch 29/500 1/1 [==============================] - 0s 0s/step - loss: 0.7422 - accuracy: 0.7000 Epoch 30/500 1/1 [==============================] - 0s 0s/step - loss: 0.7308 - accuracy: 0.7000 Epoch 31/500 1/1 [==============================] - 0s 0s/step - loss: 0.7191 - accuracy: 0.7000 Epoch 32/500 1/1 [==============================] - 0s 0s/step - loss: 0.7072 - accuracy: 0.6500 Epoch 33/500 1/1 [==============================] - 0s 8ms/step - loss: 0.6950 - accuracy: 0.7000 Epoch 34/500 1/1 [==============================] - 0s 0s/step - loss: 0.6831 - accuracy: 0.7000 Epoch 35/500 1/1 [==============================] - 0s 0s/step - loss: 0.6711 - accuracy: 0.7500 Epoch 36/500 1/1 [==============================] - 0s 8ms/step - loss: 0.6594 - accuracy: 0.7000 Epoch 37/500 1/1 [==============================] - 0s 0s/step - loss: 0.6482 - accuracy: 0.7500 Epoch 38/500 1/1 [==============================] - 0s 0s/step - loss: 0.6376 - accuracy: 0.7500 Epoch 39/500 1/1 [==============================] - 0s 0s/step - loss: 0.6278 - accuracy: 0.7500 Epoch 40/500 1/1 [==============================] - 0s 0s/step - loss: 0.6191 - accuracy: 0.7500 Epoch 41/500 1/1 [==============================] - 0s 0s/step - loss: 0.6110 - accuracy: 0.7500 Epoch 42/500 1/1 [==============================] - 0s 0s/step - loss: 0.6036 - accuracy: 0.7500 Epoch 43/500 1/1 [==============================] - 0s 0s/step - loss: 0.5968 - accuracy: 0.7500 Epoch 44/500 1/1 [==============================] - 0s 0s/step - loss: 0.5903 - accuracy: 0.7500 Epoch 45/500 1/1 [==============================] - 0s 0s/step - loss: 0.5840 - accuracy: 0.7500 Epoch 46/500 1/1 [==============================] - 0s 0s/step - loss: 0.5778 - accuracy: 0.7500 Epoch 47/500 1/1 [==============================] - 0s 0s/step - loss: 0.5714 - accuracy: 0.7500 Epoch 48/500 1/1 [==============================] - 0s 0s/step - loss: 0.5647 - accuracy: 0.7500 Epoch 49/500 1/1 [==============================] - 0s 0s/step - loss: 0.5578 - accuracy: 0.7500 Epoch 50/500 1/1 [==============================] - 0s 8ms/step - loss: 0.5507 - accuracy: 0.7500 Epoch 51/500 1/1 [==============================] - 0s 0s/step - loss: 0.5436 - accuracy: 0.7500 Epoch 52/500 1/1 [==============================] - 0s 0s/step - loss: 0.5367 - accuracy: 0.7500 Epoch 53/500 1/1 [==============================] - 0s 0s/step - loss: 0.5300 - accuracy: 0.7500 Epoch 54/500 1/1 [==============================] - 0s 0s/step - loss: 0.5236 - accuracy: 0.7500 Epoch 55/500 1/1 [==============================] - 0s 0s/step - loss: 0.5177 - accuracy: 0.7500 Epoch 56/500 1/1 [==============================] - 0s 8ms/step - loss: 0.5121 - accuracy: 0.8000 Epoch 57/500 1/1 [==============================] - 0s 0s/step - loss: 0.5070 - accuracy: 0.8500 Epoch 58/500 1/1 [==============================] - 0s 8ms/step - loss: 0.5020 - accuracy: 0.8500 Epoch 59/500 1/1 [==============================] - 0s 0s/step - loss: 0.4971 - accuracy: 0.8500 Epoch 60/500 1/1 [==============================] - 0s 0s/step - loss: 0.4922 - accuracy: 0.8500 Epoch 61/500 1/1 [==============================] - 0s 0s/step - loss: 0.4873 - accuracy: 0.8500 Epoch 62/500 1/1 [==============================] - 0s 0s/step - loss: 0.4824 - accuracy: 0.8500 Epoch 63/500 1/1 [==============================] - 0s 0s/step - loss: 0.4774 - accuracy: 0.8500 Epoch 64/500 1/1 [==============================] - 0s 0s/step - loss: 0.4728 - accuracy: 0.8500 Epoch 65/500 1/1 [==============================] - 0s 0s/step - loss: 0.4681 - accuracy: 0.8500 Epoch 66/500 1/1 [==============================] - 0s 0s/step - loss: 0.4635 - accuracy: 0.8500 Epoch 67/500 1/1 [==============================] - 0s 0s/step - loss: 0.4590 - accuracy: 0.8500 Epoch 68/500 1/1 [==============================] - 0s 0s/step - loss: 0.4543 - accuracy: 0.8500 Epoch 69/500 1/1 [==============================] - 0s 8ms/step - loss: 0.4498 - accuracy: 0.8500 Epoch 70/500 1/1 [==============================] - 0s 0s/step - loss: 0.4453 - accuracy: 0.8500 Epoch 71/500 1/1 [==============================] - 0s 0s/step - loss: 0.4408 - accuracy: 0.8000 Epoch 72/500 1/1 [==============================] - 0s 0s/step - loss: 0.4366 - accuracy: 0.8000 Epoch 73/500 1/1 [==============================] - 0s 0s/step - loss: 0.4325 - accuracy: 0.8000 Epoch 74/500 1/1 [==============================] - 0s 0s/step - loss: 0.4287 - accuracy: 0.8000 Epoch 75/500 1/1 [==============================] - 0s 0s/step - loss: 0.4248 - accuracy: 0.8000 Epoch 76/500 1/1 [==============================] - 0s 0s/step - loss: 0.4210 - accuracy: 0.8000 Epoch 77/500 1/1 [==============================] - 0s 0s/step - loss: 0.4171 - accuracy: 0.8000 Epoch 78/500 1/1 [==============================] - 0s 0s/step - loss: 0.4134 - accuracy: 0.8000 Epoch 79/500 1/1 [==============================] - 0s 8ms/step - loss: 0.4097 - accuracy: 0.8000 Epoch 80/500 1/1 [==============================] - 0s 0s/step - loss: 0.4061 - accuracy: 0.8000 Epoch 81/500 1/1 [==============================] - 0s 0s/step - loss: 0.4025 - accuracy: 0.8000 Epoch 82/500 1/1 [==============================] - 0s 8ms/step - loss: 0.3988 - accuracy: 0.8000 Epoch 83/500 1/1 [==============================] - 0s 0s/step - loss: 0.3951 - accuracy: 0.8000 Epoch 84/500 1/1 [==============================] - 0s 0s/step - loss: 0.3914 - accuracy: 0.8000 Epoch 85/500 1/1 [==============================] - 0s 0s/step - loss: 0.3878 - accuracy: 0.8000 Epoch 86/500 1/1 [==============================] - 0s 0s/step - loss: 0.3843 - accuracy: 0.8000 Epoch 87/500 1/1 [==============================] - 0s 0s/step - loss: 0.3807 - accuracy: 0.8500 Epoch 88/500 1/1 [==============================] - 0s 0s/step - loss: 0.3771 - accuracy: 0.8500 Epoch 89/500 1/1 [==============================] - 0s 0s/step - loss: 0.3736 - accuracy: 0.8500 Epoch 90/500 1/1 [==============================] - 0s 0s/step - loss: 0.3701 - accuracy: 0.8500 Epoch 91/500 1/1 [==============================] - 0s 0s/step - loss: 0.3667 - accuracy: 0.8500 Epoch 92/500 1/1 [==============================] - 0s 0s/step - loss: 0.3633 - accuracy: 0.8500 Epoch 93/500 1/1 [==============================] - 0s 8ms/step - loss: 0.3600 - accuracy: 0.8500 Epoch 94/500 1/1 [==============================] - 0s 0s/step - loss: 0.3567 - accuracy: 0.8500 Epoch 95/500 1/1 [==============================] - 0s 0s/step - loss: 0.3534 - accuracy: 0.8500 Epoch 96/500 1/1 [==============================] - 0s 8ms/step - loss: 0.3500 - accuracy: 0.9000 Epoch 97/500 1/1 [==============================] - 0s 0s/step - loss: 0.3466 - accuracy: 0.9000 Epoch 98/500 1/1 [==============================] - 0s 0s/step - loss: 0.3432 - accuracy: 0.9000 Epoch 99/500 1/1 [==============================] - 0s 0s/step - loss: 0.3400 - accuracy: 0.9000 Epoch 100/500 1/1 [==============================] - 0s 0s/step - loss: 0.3368 - accuracy: 0.9000 Epoch 101/500 1/1 [==============================] - 0s 0s/step - loss: 0.3335 - accuracy: 0.9000 Epoch 102/500 1/1 [==============================] - 0s 0s/step - loss: 0.3301 - accuracy: 0.9000 Epoch 103/500 1/1 [==============================] - 0s 0s/step - loss: 0.3269 - accuracy: 0.9000 Epoch 104/500 1/1 [==============================] - 0s 0s/step - loss: 0.3236 - accuracy: 0.9000 Epoch 105/500 1/1 [==============================] - 0s 8ms/step - loss: 0.3204 - accuracy: 0.9000 Epoch 106/500 1/1 [==============================] - 0s 0s/step - loss: 0.3172 - accuracy: 0.9000 Epoch 107/500 1/1 [==============================] - 0s 0s/step - loss: 0.3140 - accuracy: 0.9000 Epoch 108/500 1/1 [==============================] - 0s 8ms/step - loss: 0.3108 - accuracy: 0.9000 Epoch 109/500 1/1 [==============================] - 0s 0s/step - loss: 0.3077 - accuracy: 0.9000 Epoch 110/500 1/1 [==============================] - 0s 0s/step - loss: 0.3045 - accuracy: 0.9000 Epoch 111/500 1/1 [==============================] - 0s 0s/step - loss: 0.3014 - accuracy: 0.9000 Epoch 112/500 1/1 [==============================] - 0s 0s/step - loss: 0.2984 - accuracy: 0.9000 Epoch 113/500 1/1 [==============================] - 0s 0s/step - loss: 0.2955 - accuracy: 0.9000 Epoch 114/500 1/1 [==============================] - 0s 0s/step - loss: 0.2925 - accuracy: 0.9000 Epoch 115/500 1/1 [==============================] - 0s 0s/step - loss: 0.2896 - accuracy: 0.9000 Epoch 116/500 1/1 [==============================] - 0s 8ms/step - loss: 0.2867 - accuracy: 0.9000 Epoch 117/500 1/1 [==============================] - 0s 0s/step - loss: 0.2836 - accuracy: 0.9000 Epoch 118/500 1/1 [==============================] - 0s 0s/step - loss: 0.2807 - accuracy: 0.9000 Epoch 119/500 1/1 [==============================] - 0s 0s/step - loss: 0.2778 - accuracy: 0.9000 Epoch 120/500 1/1 [==============================] - 0s 0s/step - loss: 0.2750 - accuracy: 0.9000 Epoch 121/500 1/1 [==============================] - 0s 0s/step - loss: 0.2721 - accuracy: 0.9000 Epoch 122/500 1/1 [==============================] - 0s 0s/step - loss: 0.2693 - accuracy: 0.9000 Epoch 123/500 1/1 [==============================] - 0s 0s/step - loss: 0.2665 - accuracy: 0.9000 Epoch 124/500 1/1 [==============================] - 0s 8ms/step - loss: 0.2637 - accuracy: 0.9000 Epoch 125/500 1/1 [==============================] - 0s 0s/step - loss: 0.2609 - accuracy: 0.9000 Epoch 126/500 1/1 [==============================] - 0s 0s/step - loss: 0.2581 - accuracy: 0.9000 Epoch 127/500 1/1 [==============================] - 0s 0s/step - loss: 0.2553 - accuracy: 0.9000 Epoch 128/500 1/1 [==============================] - 0s 0s/step - loss: 0.2526 - accuracy: 0.9000 Epoch 129/500 1/1 [==============================] - 0s 8ms/step - loss: 0.2499 - accuracy: 0.9000 Epoch 130/500 1/1 [==============================] - 0s 0s/step - loss: 0.2472 - accuracy: 0.9000 Epoch 131/500 1/1 [==============================] - 0s 0s/step - loss: 0.2445 - accuracy: 0.9000 Epoch 132/500 1/1 [==============================] - 0s 0s/step - loss: 0.2418 - accuracy: 0.9000 Epoch 133/500 1/1 [==============================] - 0s 0s/step - loss: 0.2392 - accuracy: 0.9000 Epoch 134/500 1/1 [==============================] - 0s 0s/step - loss: 0.2366 - accuracy: 0.9000 Epoch 135/500 1/1 [==============================] - 0s 0s/step - loss: 0.2341 - accuracy: 0.9000 Epoch 136/500 1/1 [==============================] - 0s 0s/step - loss: 0.2316 - accuracy: 0.9000 Epoch 137/500 1/1 [==============================] - 0s 0s/step - loss: 0.2291 - accuracy: 0.9000 Epoch 138/500 1/1 [==============================] - 0s 0s/step - loss: 0.2266 - accuracy: 0.9000 Epoch 139/500 1/1 [==============================] - 0s 0s/step - loss: 0.2241 - accuracy: 0.9000 Epoch 140/500 1/1 [==============================] - 0s 0s/step - loss: 0.2217 - accuracy: 0.9000 Epoch 141/500 1/1 [==============================] - 0s 0s/step - loss: 0.2193 - accuracy: 1.0000 Epoch 142/500 1/1 [==============================] - 0s 0s/step - loss: 0.2170 - accuracy: 1.0000 Epoch 143/500 1/1 [==============================] - 0s 0s/step - loss: 0.2146 - accuracy: 1.0000 Epoch 144/500 1/1 [==============================] - 0s 8ms/step - loss: 0.2122 - accuracy: 1.0000 Epoch 145/500 1/1 [==============================] - 0s 0s/step - loss: 0.2099 - accuracy: 1.0000 Epoch 146/500 1/1 [==============================] - 0s 0s/step - loss: 0.2076 - accuracy: 1.0000 Epoch 147/500 1/1 [==============================] - 0s 8ms/step - loss: 0.2053 - accuracy: 1.0000 Epoch 148/500 1/1 [==============================] - 0s 0s/step - loss: 0.2031 - accuracy: 1.0000 Epoch 149/500 1/1 [==============================] - 0s 8ms/step - loss: 0.2008 - accuracy: 1.0000 Epoch 150/500 1/1 [==============================] - 0s 0s/step - loss: 0.1986 - accuracy: 1.0000 Epoch 151/500 1/1 [==============================] - 0s 0s/step - loss: 0.1964 - accuracy: 1.0000 Epoch 152/500 1/1 [==============================] - 0s 0s/step - loss: 0.1942 - accuracy: 1.0000 Epoch 153/500 1/1 [==============================] - 0s 0s/step - loss: 0.1920 - accuracy: 1.0000 Epoch 154/500 1/1 [==============================] - 0s 8ms/step - loss: 0.1899 - accuracy: 1.0000 Epoch 155/500 1/1 [==============================] - 0s 0s/step - loss: 0.1877 - accuracy: 1.0000 Epoch 156/500 1/1 [==============================] - 0s 0s/step - loss: 0.1856 - accuracy: 1.0000 Epoch 157/500 1/1 [==============================] - 0s 8ms/step - loss: 0.1835 - accuracy: 1.0000 Epoch 158/500 1/1 [==============================] - 0s 0s/step - loss: 0.1814 - accuracy: 1.0000 Epoch 159/500 1/1 [==============================] - 0s 0s/step - loss: 0.1793 - accuracy: 1.0000 Epoch 160/500 1/1 [==============================] - 0s 0s/step - loss: 0.1772 - accuracy: 1.0000 Epoch 161/500 1/1 [==============================] - 0s 0s/step - loss: 0.1752 - accuracy: 1.0000 Epoch 162/500 1/1 [==============================] - 0s 0s/step - loss: 0.1732 - accuracy: 1.0000 Epoch 163/500 1/1 [==============================] - 0s 0s/step - loss: 0.1711 - accuracy: 1.0000 Epoch 164/500 1/1 [==============================] - 0s 8ms/step - loss: 0.1691 - accuracy: 1.0000 Epoch 165/500 1/1 [==============================] - 0s 0s/step - loss: 0.1671 - accuracy: 1.0000 Epoch 166/500 1/1 [==============================] - 0s 0s/step - loss: 0.1651 - accuracy: 1.0000 Epoch 167/500 1/1 [==============================] - 0s 0s/step - loss: 0.1632 - accuracy: 1.0000 Epoch 168/500 1/1 [==============================] - 0s 0s/step - loss: 0.1612 - accuracy: 1.0000 Epoch 169/500 1/1 [==============================] - 0s 0s/step - loss: 0.1593 - accuracy: 1.0000 Epoch 170/500 1/1 [==============================] - 0s 0s/step - loss: 0.1574 - accuracy: 1.0000 Epoch 171/500 1/1 [==============================] - 0s 0s/step - loss: 0.1555 - accuracy: 1.0000 Epoch 172/500 1/1 [==============================] - 0s 0s/step - loss: 0.1536 - accuracy: 1.0000 Epoch 173/500 1/1 [==============================] - 0s 0s/step - loss: 0.1518 - accuracy: 1.0000 Epoch 174/500 1/1 [==============================] - 0s 0s/step - loss: 0.1499 - accuracy: 1.0000 Epoch 175/500 1/1 [==============================] - 0s 0s/step - loss: 0.1481 - accuracy: 1.0000 Epoch 176/500 1/1 [==============================] - 0s 0s/step - loss: 0.1463 - accuracy: 1.0000 Epoch 177/500 1/1 [==============================] - 0s 0s/step - loss: 0.1445 - accuracy: 1.0000 Epoch 178/500 1/1 [==============================] - 0s 8ms/step - loss: 0.1427 - accuracy: 1.0000 Epoch 179/500 1/1 [==============================] - 0s 0s/step - loss: 0.1409 - accuracy: 1.0000 Epoch 180/500 1/1 [==============================] - 0s 0s/step - loss: 0.1392 - accuracy: 1.0000 Epoch 181/500 1/1 [==============================] - 0s 0s/step - loss: 0.1375 - accuracy: 1.0000 Epoch 182/500 1/1 [==============================] - 0s 8ms/step - loss: 0.1357 - accuracy: 1.0000 Epoch 183/500 1/1 [==============================] - 0s 0s/step - loss: 0.1340 - accuracy: 1.0000 Epoch 184/500 1/1 [==============================] - 0s 0s/step - loss: 0.1323 - accuracy: 1.0000 Epoch 185/500 1/1 [==============================] - 0s 0s/step - loss: 0.1307 - accuracy: 1.0000 Epoch 186/500 1/1 [==============================] - 0s 8ms/step - loss: 0.1290 - accuracy: 1.0000 Epoch 187/500 1/1 [==============================] - 0s 0s/step - loss: 0.1274 - accuracy: 1.0000 Epoch 188/500 1/1 [==============================] - 0s 0s/step - loss: 0.1257 - accuracy: 1.0000 Epoch 189/500 1/1 [==============================] - 0s 0s/step - loss: 0.1241 - accuracy: 1.0000 Epoch 190/500 1/1 [==============================] - 0s 8ms/step - loss: 0.1225 - accuracy: 1.0000 Epoch 191/500 1/1 [==============================] - 0s 0s/step - loss: 0.1210 - accuracy: 1.0000 Epoch 192/500 1/1 [==============================] - 0s 0s/step - loss: 0.1194 - accuracy: 1.0000 Epoch 193/500 1/1 [==============================] - 0s 0s/step - loss: 0.1178 - accuracy: 1.0000 Epoch 194/500 1/1 [==============================] - 0s 0s/step - loss: 0.1163 - accuracy: 1.0000 Epoch 195/500 1/1 [==============================] - 0s 0s/step - loss: 0.1148 - accuracy: 1.0000 Epoch 196/500 1/1 [==============================] - 0s 0s/step - loss: 0.1133 - accuracy: 1.0000 Epoch 197/500 1/1 [==============================] - 0s 8ms/step - loss: 0.1118 - accuracy: 1.0000 Epoch 198/500 1/1 [==============================] - 0s 0s/step - loss: 0.1104 - accuracy: 1.0000 Epoch 199/500 1/1 [==============================] - 0s 0s/step - loss: 0.1089 - accuracy: 1.0000 Epoch 200/500 1/1 [==============================] - 0s 0s/step - loss: 0.1075 - accuracy: 1.0000 Epoch 201/500 1/1 [==============================] - 0s 8ms/step - loss: 0.1061 - accuracy: 1.0000 Epoch 202/500 1/1 [==============================] - 0s 8ms/step - loss: 0.1047 - accuracy: 1.0000 Epoch 203/500 1/1 [==============================] - 0s 0s/step - loss: 0.1033 - accuracy: 1.0000 Epoch 204/500 1/1 [==============================] - 0s 0s/step - loss: 0.1020 - accuracy: 1.0000 Epoch 205/500 1/1 [==============================] - 0s 0s/step - loss: 0.1006 - accuracy: 1.0000 Epoch 206/500 1/1 [==============================] - 0s 0s/step - loss: 0.0993 - accuracy: 1.0000 Epoch 207/500 1/1 [==============================] - 0s 0s/step - loss: 0.0980 - accuracy: 1.0000 Epoch 208/500 1/1 [==============================] - 0s 0s/step - loss: 0.0967 - accuracy: 1.0000 Epoch 209/500 1/1 [==============================] - 0s 0s/step - loss: 0.0954 - accuracy: 1.0000 Epoch 210/500 1/1 [==============================] - 0s 0s/step - loss: 0.0941 - accuracy: 1.0000 Epoch 211/500 1/1 [==============================] - 0s 0s/step - loss: 0.0929 - accuracy: 1.0000 Epoch 212/500 1/1 [==============================] - 0s 0s/step - loss: 0.0917 - accuracy: 1.0000 Epoch 213/500 1/1 [==============================] - 0s 0s/step - loss: 0.0905 - accuracy: 1.0000 Epoch 214/500 1/1 [==============================] - 0s 0s/step - loss: 0.0893 - accuracy: 1.0000 Epoch 215/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0881 - accuracy: 1.0000 Epoch 216/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0870 - accuracy: 1.0000 Epoch 217/500 1/1 [==============================] - 0s 0s/step - loss: 0.0859 - accuracy: 1.0000 Epoch 218/500 1/1 [==============================] - 0s 0s/step - loss: 0.0847 - accuracy: 1.0000 Epoch 219/500 1/1 [==============================] - 0s 0s/step - loss: 0.0836 - accuracy: 1.0000 Epoch 220/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0825 - accuracy: 1.0000 Epoch 221/500 1/1 [==============================] - 0s 0s/step - loss: 0.0815 - accuracy: 1.0000 Epoch 222/500 1/1 [==============================] - 0s 0s/step - loss: 0.0805 - accuracy: 1.0000 Epoch 223/500 1/1 [==============================] - 0s 0s/step - loss: 0.0794 - accuracy: 1.0000 Epoch 224/500 1/1 [==============================] - 0s 0s/step - loss: 0.0784 - accuracy: 1.0000 Epoch 225/500 1/1 [==============================] - 0s 0s/step - loss: 0.0774 - accuracy: 1.0000 Epoch 226/500 1/1 [==============================] - 0s 0s/step - loss: 0.0764 - accuracy: 1.0000 Epoch 227/500 1/1 [==============================] - 0s 0s/step - loss: 0.0754 - accuracy: 1.0000 Epoch 228/500 1/1 [==============================] - 0s 0s/step - loss: 0.0745 - accuracy: 1.0000 Epoch 229/500 1/1 [==============================] - 0s 0s/step - loss: 0.0735 - accuracy: 1.0000 Epoch 230/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0726 - accuracy: 1.0000 Epoch 231/500 1/1 [==============================] - 0s 0s/step - loss: 0.0717 - accuracy: 1.0000 Epoch 232/500 1/1 [==============================] - 0s 0s/step - loss: 0.0707 - accuracy: 1.0000 Epoch 233/500 1/1 [==============================] - 0s 0s/step - loss: 0.0698 - accuracy: 1.0000 Epoch 234/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0690 - accuracy: 1.0000 Epoch 235/500 1/1 [==============================] - 0s 0s/step - loss: 0.0681 - accuracy: 1.0000 Epoch 236/500 1/1 [==============================] - 0s 0s/step - loss: 0.0672 - accuracy: 1.0000 Epoch 237/500 1/1 [==============================] - 0s 0s/step - loss: 0.0663 - accuracy: 1.0000 Epoch 238/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0655 - accuracy: 1.0000 Epoch 239/500 1/1 [==============================] - 0s 0s/step - loss: 0.0647 - accuracy: 1.0000 Epoch 240/500 1/1 [==============================] - 0s 0s/step - loss: 0.0638 - accuracy: 1.0000 Epoch 241/500 1/1 [==============================] - 0s 0s/step - loss: 0.0630 - accuracy: 1.0000 Epoch 242/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0622 - accuracy: 1.0000 Epoch 243/500 1/1 [==============================] - 0s 0s/step - loss: 0.0615 - accuracy: 1.0000 Epoch 244/500 1/1 [==============================] - 0s 0s/step - loss: 0.0607 - accuracy: 1.0000 Epoch 245/500 1/1 [==============================] - 0s 0s/step - loss: 0.0599 - accuracy: 1.0000 Epoch 246/500 1/1 [==============================] - 0s 0s/step - loss: 0.0592 - accuracy: 1.0000 Epoch 247/500 1/1 [==============================] - 0s 0s/step - loss: 0.0584 - accuracy: 1.0000 Epoch 248/500 1/1 [==============================] - 0s 0s/step - loss: 0.0577 - accuracy: 1.0000 Epoch 249/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0570 - accuracy: 1.0000 Epoch 250/500 1/1 [==============================] - 0s 0s/step - loss: 0.0563 - accuracy: 1.0000 Epoch 251/500 1/1 [==============================] - 0s 0s/step - loss: 0.0556 - accuracy: 1.0000 Epoch 252/500 1/1 [==============================] - 0s 0s/step - loss: 0.0549 - accuracy: 1.0000 Epoch 253/500 1/1 [==============================] - 0s 0s/step - loss: 0.0543 - accuracy: 1.0000 Epoch 254/500 1/1 [==============================] - 0s 0s/step - loss: 0.0536 - accuracy: 1.0000 Epoch 255/500 1/1 [==============================] - 0s 0s/step - loss: 0.0530 - accuracy: 1.0000 Epoch 256/500 1/1 [==============================] - 0s 0s/step - loss: 0.0523 - accuracy: 1.0000 Epoch 257/500 1/1 [==============================] - 0s 0s/step - loss: 0.0517 - accuracy: 1.0000 Epoch 258/500 1/1 [==============================] - 0s 0s/step - loss: 0.0511 - accuracy: 1.0000 Epoch 259/500 1/1 [==============================] - 0s 0s/step - loss: 0.0505 - accuracy: 1.0000 Epoch 260/500 1/1 [==============================] - 0s 0s/step - loss: 0.0499 - accuracy: 1.0000 Epoch 261/500 1/1 [==============================] - 0s 0s/step - loss: 0.0493 - accuracy: 1.0000 Epoch 262/500 1/1 [==============================] - 0s 0s/step - loss: 0.0487 - accuracy: 1.0000 Epoch 263/500 1/1 [==============================] - 0s 0s/step - loss: 0.0481 - accuracy: 1.0000 Epoch 264/500 1/1 [==============================] - 0s 0s/step - loss: 0.0476 - accuracy: 1.0000 Epoch 265/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0470 - accuracy: 1.0000 Epoch 266/500 1/1 [==============================] - 0s 0s/step - loss: 0.0465 - accuracy: 1.0000 Epoch 267/500 1/1 [==============================] - 0s 0s/step - loss: 0.0459 - accuracy: 1.0000 Epoch 268/500 1/1 [==============================] - 0s 0s/step - loss: 0.0454 - accuracy: 1.0000 Epoch 269/500 1/1 [==============================] - 0s 0s/step - loss: 0.0449 - accuracy: 1.0000 Epoch 270/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0444 - accuracy: 1.0000 Epoch 271/500 1/1 [==============================] - 0s 0s/step - loss: 0.0439 - accuracy: 1.0000 Epoch 272/500 1/1 [==============================] - 0s 0s/step - loss: 0.0434 - accuracy: 1.0000 Epoch 273/500 1/1 [==============================] - 0s 0s/step - loss: 0.0429 - accuracy: 1.0000 Epoch 274/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0424 - accuracy: 1.0000 Epoch 275/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0420 - accuracy: 1.0000 Epoch 276/500 1/1 [==============================] - 0s 0s/step - loss: 0.0415 - accuracy: 1.0000 Epoch 277/500 1/1 [==============================] - 0s 0s/step - loss: 0.0410 - accuracy: 1.0000 Epoch 278/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0406 - accuracy: 1.0000 Epoch 279/500 1/1 [==============================] - 0s 0s/step - loss: 0.0401 - accuracy: 1.0000 Epoch 280/500 1/1 [==============================] - 0s 0s/step - loss: 0.0397 - accuracy: 1.0000 Epoch 281/500 1/1 [==============================] - 0s 0s/step - loss: 0.0393 - accuracy: 1.0000 Epoch 282/500 1/1 [==============================] - 0s 0s/step - loss: 0.0389 - accuracy: 1.0000 Epoch 283/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0384 - accuracy: 1.0000 Epoch 284/500 1/1 [==============================] - 0s 0s/step - loss: 0.0380 - accuracy: 1.0000 Epoch 285/500 1/1 [==============================] - 0s 0s/step - loss: 0.0376 - accuracy: 1.0000 Epoch 286/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0372 - accuracy: 1.0000 Epoch 287/500 1/1 [==============================] - 0s 0s/step - loss: 0.0368 - accuracy: 1.0000 Epoch 288/500 1/1 [==============================] - 0s 0s/step - loss: 0.0364 - accuracy: 1.0000 Epoch 289/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0361 - accuracy: 1.0000 Epoch 290/500 1/1 [==============================] - 0s 0s/step - loss: 0.0357 - accuracy: 1.0000 Epoch 291/500 1/1 [==============================] - 0s 0s/step - loss: 0.0353 - accuracy: 1.0000 Epoch 292/500 1/1 [==============================] - 0s 0s/step - loss: 0.0350 - accuracy: 1.0000 Epoch 293/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0346 - accuracy: 1.0000 Epoch 294/500 1/1 [==============================] - 0s 0s/step - loss: 0.0342 - accuracy: 1.0000 Epoch 295/500 1/1 [==============================] - 0s 0s/step - loss: 0.0339 - accuracy: 1.0000 Epoch 296/500 1/1 [==============================] - 0s 0s/step - loss: 0.0336 - accuracy: 1.0000 Epoch 297/500 1/1 [==============================] - 0s 0s/step - loss: 0.0332 - accuracy: 1.0000 Epoch 298/500 1/1 [==============================] - 0s 0s/step - loss: 0.0329 - accuracy: 1.0000 Epoch 299/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0326 - accuracy: 1.0000 Epoch 300/500 1/1 [==============================] - 0s 0s/step - loss: 0.0322 - accuracy: 1.0000 Epoch 301/500 1/1 [==============================] - 0s 0s/step - loss: 0.0319 - accuracy: 1.0000 Epoch 302/500 1/1 [==============================] - 0s 0s/step - loss: 0.0316 - accuracy: 1.0000 Epoch 303/500 1/1 [==============================] - 0s 0s/step - loss: 0.0313 - accuracy: 1.0000 Epoch 304/500 1/1 [==============================] - 0s 0s/step - loss: 0.0310 - accuracy: 1.0000 Epoch 305/500 1/1 [==============================] - 0s 0s/step - loss: 0.0307 - accuracy: 1.0000 Epoch 306/500 1/1 [==============================] - 0s 0s/step - loss: 0.0304 - accuracy: 1.0000 Epoch 307/500 1/1 [==============================] - 0s 0s/step - loss: 0.0301 - accuracy: 1.0000 Epoch 308/500 1/1 [==============================] - 0s 0s/step - loss: 0.0298 - accuracy: 1.0000 Epoch 309/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0296 - accuracy: 1.0000 Epoch 310/500 1/1 [==============================] - 0s 0s/step - loss: 0.0293 - accuracy: 1.0000 Epoch 311/500 1/1 [==============================] - 0s 0s/step - loss: 0.0290 - accuracy: 1.0000 Epoch 312/500 1/1 [==============================] - 0s 0s/step - loss: 0.0287 - accuracy: 1.0000 Epoch 313/500 1/1 [==============================] - 0s 0s/step - loss: 0.0285 - accuracy: 1.0000 Epoch 314/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0282 - accuracy: 1.0000 Epoch 315/500 1/1 [==============================] - 0s 0s/step - loss: 0.0280 - accuracy: 1.0000 Epoch 316/500 1/1 [==============================] - 0s 0s/step - loss: 0.0277 - accuracy: 1.0000 Epoch 317/500 1/1 [==============================] - 0s 0s/step - loss: 0.0274 - accuracy: 1.0000 Epoch 318/500 1/1 [==============================] - 0s 0s/step - loss: 0.0272 - accuracy: 1.0000 Epoch 319/500 1/1 [==============================] - 0s 0s/step - loss: 0.0270 - accuracy: 1.0000 Epoch 320/500 1/1 [==============================] - 0s 0s/step - loss: 0.0267 - accuracy: 1.0000 Epoch 321/500 1/1 [==============================] - 0s 0s/step - loss: 0.0265 - accuracy: 1.0000 Epoch 322/500 1/1 [==============================] - 0s 0s/step - loss: 0.0262 - accuracy: 1.0000 Epoch 323/500 1/1 [==============================] - 0s 0s/step - loss: 0.0260 - accuracy: 1.0000 Epoch 324/500 1/1 [==============================] - 0s 0s/step - loss: 0.0258 - accuracy: 1.0000 Epoch 325/500 1/1 [==============================] - 0s 0s/step - loss: 0.0256 - accuracy: 1.0000 Epoch 326/500 1/1 [==============================] - 0s 0s/step - loss: 0.0253 - accuracy: 1.0000 Epoch 327/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0251 - accuracy: 1.0000 Epoch 328/500 1/1 [==============================] - 0s 0s/step - loss: 0.0249 - accuracy: 1.0000 Epoch 329/500 1/1 [==============================] - 0s 0s/step - loss: 0.0247 - accuracy: 1.0000 Epoch 330/500 1/1 [==============================] - 0s 0s/step - loss: 0.0245 - accuracy: 1.0000 Epoch 331/500 1/1 [==============================] - 0s 0s/step - loss: 0.0243 - accuracy: 1.0000 Epoch 332/500 1/1 [==============================] - 0s 0s/step - loss: 0.0241 - accuracy: 1.0000 Epoch 333/500 1/1 [==============================] - 0s 0s/step - loss: 0.0239 - accuracy: 1.0000 Epoch 334/500 1/1 [==============================] - 0s 0s/step - loss: 0.0237 - accuracy: 1.0000 Epoch 335/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0235 - accuracy: 1.0000 Epoch 336/500 1/1 [==============================] - 0s 0s/step - loss: 0.0233 - accuracy: 1.0000 Epoch 337/500 1/1 [==============================] - 0s 0s/step - loss: 0.0231 - accuracy: 1.0000 Epoch 338/500 1/1 [==============================] - 0s 0s/step - loss: 0.0229 - accuracy: 1.0000 Epoch 339/500 1/1 [==============================] - 0s 0s/step - loss: 0.0227 - accuracy: 1.0000 Epoch 340/500 1/1 [==============================] - 0s 0s/step - loss: 0.0225 - accuracy: 1.0000 Epoch 341/500 1/1 [==============================] - 0s 0s/step - loss: 0.0224 - accuracy: 1.0000 Epoch 342/500 1/1 [==============================] - 0s 0s/step - loss: 0.0222 - accuracy: 1.0000 Epoch 343/500 1/1 [==============================] - 0s 0s/step - loss: 0.0220 - accuracy: 1.0000 Epoch 344/500 1/1 [==============================] - 0s 0s/step - loss: 0.0218 - accuracy: 1.0000 Epoch 345/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0217 - accuracy: 1.0000 Epoch 346/500 1/1 [==============================] - 0s 0s/step - loss: 0.0215 - accuracy: 1.0000 Epoch 347/500 1/1 [==============================] - 0s 0s/step - loss: 0.0213 - accuracy: 1.0000 Epoch 348/500 1/1 [==============================] - 0s 0s/step - loss: 0.0211 - accuracy: 1.0000 Epoch 349/500 1/1 [==============================] - 0s 0s/step - loss: 0.0210 - accuracy: 1.0000 Epoch 350/500 1/1 [==============================] - 0s 0s/step - loss: 0.0208 - accuracy: 1.0000 Epoch 351/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0207 - accuracy: 1.0000 Epoch 352/500 1/1 [==============================] - 0s 0s/step - loss: 0.0205 - accuracy: 1.0000 Epoch 353/500 1/1 [==============================] - 0s 0s/step - loss: 0.0203 - accuracy: 1.0000 Epoch 354/500 1/1 [==============================] - 0s 0s/step - loss: 0.0202 - accuracy: 1.0000 Epoch 355/500 1/1 [==============================] - 0s 0s/step - loss: 0.0200 - accuracy: 1.0000 Epoch 356/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0199 - accuracy: 1.0000 Epoch 357/500 1/1 [==============================] - 0s 0s/step - loss: 0.0197 - accuracy: 1.0000 Epoch 358/500 1/1 [==============================] - 0s 0s/step - loss: 0.0196 - accuracy: 1.0000 Epoch 359/500 1/1 [==============================] - 0s 0s/step - loss: 0.0194 - accuracy: 1.0000 Epoch 360/500 1/1 [==============================] - 0s 0s/step - loss: 0.0193 - accuracy: 1.0000 Epoch 361/500 1/1 [==============================] - 0s 0s/step - loss: 0.0192 - accuracy: 1.0000 Epoch 362/500 1/1 [==============================] - 0s 0s/step - loss: 0.0190 - accuracy: 1.0000 Epoch 363/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0189 - accuracy: 1.0000 Epoch 364/500 1/1 [==============================] - 0s 0s/step - loss: 0.0187 - accuracy: 1.0000 Epoch 365/500 1/1 [==============================] - 0s 0s/step - loss: 0.0186 - accuracy: 1.0000 Epoch 366/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0185 - accuracy: 1.0000 Epoch 367/500 1/1 [==============================] - 0s 0s/step - loss: 0.0183 - accuracy: 1.0000 Epoch 368/500 1/1 [==============================] - 0s 0s/step - loss: 0.0182 - accuracy: 1.0000 Epoch 369/500 1/1 [==============================] - 0s 0s/step - loss: 0.0181 - accuracy: 1.0000 Epoch 370/500 1/1 [==============================] - 0s 0s/step - loss: 0.0180 - accuracy: 1.0000 Epoch 371/500 1/1 [==============================] - 0s 0s/step - loss: 0.0178 - accuracy: 1.0000 Epoch 372/500 1/1 [==============================] - 0s 0s/step - loss: 0.0177 - accuracy: 1.0000 Epoch 373/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0176 - accuracy: 1.0000 Epoch 374/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0175 - accuracy: 1.0000 Epoch 375/500 1/1 [==============================] - 0s 0s/step - loss: 0.0173 - accuracy: 1.0000 Epoch 376/500 1/1 [==============================] - 0s 0s/step - loss: 0.0172 - accuracy: 1.0000 Epoch 377/500 1/1 [==============================] - 0s 0s/step - loss: 0.0171 - accuracy: 1.0000 Epoch 378/500 1/1 [==============================] - 0s 0s/step - loss: 0.0170 - accuracy: 1.0000 Epoch 379/500 1/1 [==============================] - 0s 0s/step - loss: 0.0169 - accuracy: 1.0000 Epoch 380/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0167 - accuracy: 1.0000 Epoch 381/500 1/1 [==============================] - 0s 0s/step - loss: 0.0166 - accuracy: 1.0000 Epoch 382/500 1/1 [==============================] - 0s 0s/step - loss: 0.0165 - accuracy: 1.0000 Epoch 383/500 1/1 [==============================] - 0s 0s/step - loss: 0.0164 - accuracy: 1.0000 Epoch 384/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0163 - accuracy: 1.0000 Epoch 385/500 1/1 [==============================] - 0s 0s/step - loss: 0.0162 - accuracy: 1.0000 Epoch 386/500 1/1 [==============================] - 0s 0s/step - loss: 0.0161 - accuracy: 1.0000 Epoch 387/500 1/1 [==============================] - 0s 0s/step - loss: 0.0160 - accuracy: 1.0000 Epoch 388/500 1/1 [==============================] - 0s 0s/step - loss: 0.0159 - accuracy: 1.0000 Epoch 389/500 1/1 [==============================] - 0s 0s/step - loss: 0.0158 - accuracy: 1.0000 Epoch 390/500 1/1 [==============================] - 0s 0s/step - loss: 0.0157 - accuracy: 1.0000 Epoch 391/500 1/1 [==============================] - 0s 0s/step - loss: 0.0156 - accuracy: 1.0000 Epoch 392/500 1/1 [==============================] - 0s 0s/step - loss: 0.0155 - accuracy: 1.0000 Epoch 393/500 1/1 [==============================] - 0s 0s/step - loss: 0.0154 - accuracy: 1.0000 Epoch 394/500 1/1 [==============================] - 0s 0s/step - loss: 0.0153 - accuracy: 1.0000 Epoch 395/500 1/1 [==============================] - 0s 0s/step - loss: 0.0152 - accuracy: 1.0000 Epoch 396/500 1/1 [==============================] - 0s 0s/step - loss: 0.0151 - accuracy: 1.0000 Epoch 397/500 1/1 [==============================] - 0s 0s/step - loss: 0.0150 - accuracy: 1.0000 Epoch 398/500 1/1 [==============================] - 0s 0s/step - loss: 0.0149 - accuracy: 1.0000 Epoch 399/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0148 - accuracy: 1.0000 Epoch 400/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0147 - accuracy: 1.0000 Epoch 401/500 1/1 [==============================] - 0s 0s/step - loss: 0.0146 - accuracy: 1.0000 Epoch 402/500 1/1 [==============================] - 0s 0s/step - loss: 0.0145 - accuracy: 1.0000 Epoch 403/500 1/1 [==============================] - 0s 0s/step - loss: 0.0144 - accuracy: 1.0000 Epoch 404/500 1/1 [==============================] - 0s 0s/step - loss: 0.0143 - accuracy: 1.0000 Epoch 405/500 1/1 [==============================] - 0s 0s/step - loss: 0.0143 - accuracy: 1.0000 Epoch 406/500 1/1 [==============================] - 0s 0s/step - loss: 0.0142 - accuracy: 1.0000 Epoch 407/500 1/1 [==============================] - 0s 0s/step - loss: 0.0141 - accuracy: 1.0000 Epoch 408/500 1/1 [==============================] - 0s 0s/step - loss: 0.0140 - accuracy: 1.0000 Epoch 409/500 1/1 [==============================] - 0s 0s/step - loss: 0.0139 - accuracy: 1.0000 Epoch 410/500 1/1 [==============================] - 0s 0s/step - loss: 0.0138 - accuracy: 1.0000 Epoch 411/500 1/1 [==============================] - 0s 0s/step - loss: 0.0138 - accuracy: 1.0000 Epoch 412/500 1/1 [==============================] - 0s 0s/step - loss: 0.0137 - accuracy: 1.0000 Epoch 413/500 1/1 [==============================] - 0s 0s/step - loss: 0.0136 - accuracy: 1.0000 Epoch 414/500 1/1 [==============================] - 0s 0s/step - loss: 0.0135 - accuracy: 1.0000 Epoch 415/500 1/1 [==============================] - 0s 0s/step - loss: 0.0134 - accuracy: 1.0000 Epoch 416/500 1/1 [==============================] - 0s 0s/step - loss: 0.0134 - accuracy: 1.0000 Epoch 417/500 1/1 [==============================] - 0s 0s/step - loss: 0.0133 - accuracy: 1.0000 Epoch 418/500 1/1 [==============================] - 0s 0s/step - loss: 0.0132 - accuracy: 1.0000 Epoch 419/500 1/1 [==============================] - 0s 0s/step - loss: 0.0131 - accuracy: 1.0000 Epoch 420/500 1/1 [==============================] - 0s 0s/step - loss: 0.0130 - accuracy: 1.0000 Epoch 421/500 1/1 [==============================] - 0s 0s/step - loss: 0.0130 - accuracy: 1.0000 Epoch 422/500 1/1 [==============================] - 0s 0s/step - loss: 0.0129 - accuracy: 1.0000 Epoch 423/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0128 - accuracy: 1.0000 Epoch 424/500 1/1 [==============================] - 0s 0s/step - loss: 0.0128 - accuracy: 1.0000 Epoch 425/500 1/1 [==============================] - 0s 0s/step - loss: 0.0127 - accuracy: 1.0000 Epoch 426/500 1/1 [==============================] - 0s 0s/step - loss: 0.0126 - accuracy: 1.0000 Epoch 427/500 1/1 [==============================] - 0s 0s/step - loss: 0.0125 - accuracy: 1.0000 Epoch 428/500 1/1 [==============================] - 0s 0s/step - loss: 0.0125 - accuracy: 1.0000 Epoch 429/500 1/1 [==============================] - 0s 0s/step - loss: 0.0124 - accuracy: 1.0000 Epoch 430/500 1/1 [==============================] - 0s 0s/step - loss: 0.0123 - accuracy: 1.0000 Epoch 431/500 1/1 [==============================] - 0s 0s/step - loss: 0.0123 - accuracy: 1.0000 Epoch 432/500 1/1 [==============================] - 0s 0s/step - loss: 0.0122 - accuracy: 1.0000 Epoch 433/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0121 - accuracy: 1.0000 Epoch 434/500 1/1 [==============================] - 0s 0s/step - loss: 0.0121 - accuracy: 1.0000 Epoch 435/500 1/1 [==============================] - 0s 0s/step - loss: 0.0120 - accuracy: 1.0000 Epoch 436/500 1/1 [==============================] - 0s 0s/step - loss: 0.0119 - accuracy: 1.0000 Epoch 437/500 1/1 [==============================] - 0s 0s/step - loss: 0.0119 - accuracy: 1.0000 Epoch 438/500 1/1 [==============================] - 0s 0s/step - loss: 0.0118 - accuracy: 1.0000 Epoch 439/500 1/1 [==============================] - 0s 0s/step - loss: 0.0117 - accuracy: 1.0000 Epoch 440/500 1/1 [==============================] - 0s 0s/step - loss: 0.0117 - accuracy: 1.0000 Epoch 441/500 1/1 [==============================] - 0s 0s/step - loss: 0.0116 - accuracy: 1.0000 Epoch 442/500 1/1 [==============================] - 0s 0s/step - loss: 0.0116 - accuracy: 1.0000 Epoch 443/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0115 - accuracy: 1.0000 Epoch 444/500 1/1 [==============================] - 0s 0s/step - loss: 0.0114 - accuracy: 1.0000 Epoch 445/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0114 - accuracy: 1.0000 Epoch 446/500 1/1 [==============================] - 0s 0s/step - loss: 0.0113 - accuracy: 1.0000 Epoch 447/500 1/1 [==============================] - 0s 0s/step - loss: 0.0113 - accuracy: 1.0000 Epoch 448/500 1/1 [==============================] - 0s 0s/step - loss: 0.0112 - accuracy: 1.0000 Epoch 449/500 1/1 [==============================] - 0s 0s/step - loss: 0.0111 - accuracy: 1.0000 Epoch 450/500 1/1 [==============================] - 0s 0s/step - loss: 0.0111 - accuracy: 1.0000 Epoch 451/500 1/1 [==============================] - 0s 0s/step - loss: 0.0110 - accuracy: 1.0000 Epoch 452/500 1/1 [==============================] - 0s 0s/step - loss: 0.0110 - accuracy: 1.0000 Epoch 453/500 1/1 [==============================] - 0s 0s/step - loss: 0.0109 - accuracy: 1.0000 Epoch 454/500 1/1 [==============================] - 0s 0s/step - loss: 0.0109 - accuracy: 1.0000 Epoch 455/500 1/1 [==============================] - 0s 0s/step - loss: 0.0108 - accuracy: 1.0000 Epoch 456/500 1/1 [==============================] - 0s 0s/step - loss: 0.0108 - accuracy: 1.0000 Epoch 457/500 1/1 [==============================] - 0s 0s/step - loss: 0.0107 - accuracy: 1.0000 Epoch 458/500 1/1 [==============================] - 0s 0s/step - loss: 0.0106 - accuracy: 1.0000 Epoch 459/500 1/1 [==============================] - 0s 0s/step - loss: 0.0106 - accuracy: 1.0000 Epoch 460/500 1/1 [==============================] - 0s 0s/step - loss: 0.0105 - accuracy: 1.0000 Epoch 461/500 1/1 [==============================] - 0s 0s/step - loss: 0.0105 - accuracy: 1.0000 Epoch 462/500 1/1 [==============================] - 0s 0s/step - loss: 0.0104 - accuracy: 1.0000 Epoch 463/500 1/1 [==============================] - 0s 0s/step - loss: 0.0104 - accuracy: 1.0000 Epoch 464/500 1/1 [==============================] - 0s 0s/step - loss: 0.0103 - accuracy: 1.0000 Epoch 465/500 1/1 [==============================] - 0s 0s/step - loss: 0.0103 - accuracy: 1.0000 Epoch 466/500 1/1 [==============================] - 0s 0s/step - loss: 0.0102 - accuracy: 1.0000 Epoch 467/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0102 - accuracy: 1.0000 Epoch 468/500 1/1 [==============================] - 0s 0s/step - loss: 0.0101 - accuracy: 1.0000 Epoch 469/500 1/1 [==============================] - 0s 0s/step - loss: 0.0101 - accuracy: 1.0000 Epoch 470/500 1/1 [==============================] - 0s 0s/step - loss: 0.0100 - accuracy: 1.0000 Epoch 471/500 1/1 [==============================] - 0s 0s/step - loss: 0.0100 - accuracy: 1.0000 Epoch 472/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0099 - accuracy: 1.0000 Epoch 473/500 1/1 [==============================] - 0s 0s/step - loss: 0.0099 - accuracy: 1.0000 Epoch 474/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0099 - accuracy: 1.0000 Epoch 475/500 1/1 [==============================] - 0s 0s/step - loss: 0.0098 - accuracy: 1.0000 Epoch 476/500 1/1 [==============================] - 0s 0s/step - loss: 0.0098 - accuracy: 1.0000 Epoch 477/500 1/1 [==============================] - 0s 0s/step - loss: 0.0097 - accuracy: 1.0000 Epoch 478/500 1/1 [==============================] - 0s 0s/step - loss: 0.0097 - accuracy: 1.0000 Epoch 479/500 1/1 [==============================] - 0s 8ms/step - loss: 0.0096 - accuracy: 1.0000 Epoch 480/500 1/1 [==============================] - 0s 0s/step - loss: 0.0096 - accuracy: 1.0000 Epoch 481/500 1/1 [==============================] - 0s 0s/step - loss: 0.0095 - accuracy: 1.0000 Epoch 482/500 1/1 [==============================] - 0s 0s/step - loss: 0.0095 - accuracy: 1.0000 Epoch 483/500 1/1 [==============================] - 0s 0s/step - loss: 0.0095 - accuracy: 1.0000 Epoch 484/500 1/1 [==============================] - 0s 0s/step - loss: 0.0094 - accuracy: 1.0000 Epoch 485/500 1/1 [==============================] - 0s 0s/step - loss: 0.0094 - accuracy: 1.0000 Epoch 486/500 1/1 [==============================] - 0s 0s/step - loss: 0.0093 - accuracy: 1.0000 Epoch 487/500 1/1 [==============================] - 0s 0s/step - loss: 0.0093 - accuracy: 1.0000 Epoch 488/500 1/1 [==============================] - 0s 0s/step - loss: 0.0092 - accuracy: 1.0000 Epoch 489/500 1/1 [==============================] - 0s 0s/step - loss: 0.0092 - accuracy: 1.0000 Epoch 490/500 1/1 [==============================] - 0s 0s/step - loss: 0.0092 - accuracy: 1.0000 Epoch 491/500 1/1 [==============================] - 0s 0s/step - loss: 0.0091 - accuracy: 1.0000 Epoch 492/500 1/1 [==============================] - 0s 0s/step - loss: 0.0091 - accuracy: 1.0000 Epoch 493/500 1/1 [==============================] - 0s 0s/step - loss: 0.0090 - accuracy: 1.0000 Epoch 494/500 1/1 [==============================] - 0s 0s/step - loss: 0.0090 - accuracy: 1.0000 Epoch 495/500 1/1 [==============================] - 0s 0s/step - loss: 0.0090 - accuracy: 1.0000 Epoch 496/500 1/1 [==============================] - 0s 0s/step - loss: 0.0089 - accuracy: 1.0000 Epoch 497/500 1/1 [==============================] - 0s 0s/step - loss: 0.0089 - accuracy: 1.0000 Epoch 498/500 1/1 [==============================] - 0s 0s/step - loss: 0.0088 - accuracy: 1.0000 Epoch 499/500 1/1 [==============================] - 0s 0s/step - loss: 0.0088 - accuracy: 1.0000 Epoch 500/500 1/1 [==============================] - 0s 0s/step - loss: 0.0088 - accuracy: 1.0000
lr = 0.00001
Lambda = 1e3
train_and_test_loop(1, lr, Lambda)
1/1 [==============================] - 0s 8ms/step - loss: 9411.3271 - accuracy: 0.0288
lr = 0.00001
Lambda = 0
train_and_test_loop(1, lr, Lambda)
1/1 [==============================] - 0s 0s/step - loss: 1.8391 - accuracy: 0.2315
def train_and_test_loop1(iterations, lr, Lambda, verb=True):
score = {}
## hyperparameters
iterations = iterations
learning_rate = lr
hidden_node1 = 26
hidden_node2 = 14
output_nodes = 6
model = Sequential()
model.add(Dense(hidden_node1, input_shape=(11,), activation='relu'))
model.add(Dense(hidden_node2, activation='relu'))
model.add(Dense(output_nodes, activation='softmax', kernel_regularizer=regularizers.l2(Lambda)))
sgd = optimizers.SGD(lr=learning_rate, decay=1e-6, momentum=0.9)
# Compile model
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['AUC'])
# Fit the model
model.fit(np.array(X_train), y_train, epochs=iterations, batch_size=1000, verbose=0)
score["train"] = model.evaluate(np.array(X_train), y_train, verbose=0)
score["val"] = model.evaluate(np.array(X_val), y_val, verbose=0)
return score
# Creating an evaluation Dataframe for further use
df_eval = pd.DataFrame(columns=["LR", "Lambda", "train_loss", "train_accuracy", "val_loss", "val_accuracy"])
import math
for k in range(1,10):
lr = math.pow(10, np.random.uniform(-2, -1))
Lambda = 3.1478848403116417e-06
best_acc = train_and_test_loop1(100, lr, Lambda, False)
print("Try {0}/{1}: Best_val_acc: {2},\n lr: {3}, Lambda: {4}".format(k, 9, best_acc, lr, Lambda))
df_eval.loc[len(df_eval)] = [lr, Lambda, best_acc["train"][0], best_acc["train"][1],
best_acc["val"][0], best_acc["val"][1]]
Try 1/9: Best_val_acc: {'train': [0.9688308238983154, 0.9044107794761658], 'val': [1.0201138257980347, 0.890222430229187]},
lr: 0.014348159897365887, Lambda: 3.1478848403116417e-06
Try 2/9: Best_val_acc: {'train': [0.9711223244667053, 0.902430534362793], 'val': [1.041279911994934, 0.8853806257247925]},
lr: 0.016442724939477668, Lambda: 3.1478848403116417e-06
Try 3/9: Best_val_acc: {'train': [0.8739877343177795, 0.9200472235679626], 'val': [0.9909087419509888, 0.8968706130981445]},
lr: 0.06372093096397173, Lambda: 3.1478848403116417e-06
Try 4/9: Best_val_acc: {'train': [0.8931687474250793, 0.9168083071708679], 'val': [0.9833325147628784, 0.8989100456237793]},
lr: 0.06357205192716267, Lambda: 3.1478848403116417e-06
Try 5/9: Best_val_acc: {'train': [0.9894654154777527, 0.898215651512146], 'val': [1.0403515100479126, 0.8824206590652466]},
lr: 0.011785503095166714, Lambda: 3.1478848403116417e-06
Try 6/9: Best_val_acc: {'train': [0.8553031086921692, 0.9235700368881226], 'val': [0.9991909861564636, 0.8984987735748291]},
lr: 0.06396529358522594, Lambda: 3.1478848403116417e-06
Try 7/9: Best_val_acc: {'train': [0.9625827074050903, 0.9030699729919434], 'val': [1.0130378007888794, 0.8916305899620056]},
lr: 0.015296268384686914, Lambda: 3.1478848403116417e-06
Try 8/9: Best_val_acc: {'train': [0.8860230445861816, 0.9192364811897278], 'val': [0.9824147820472717, 0.8986973762512207]},
lr: 0.05108248166561557, Lambda: 3.1478848403116417e-06
Try 9/9: Best_val_acc: {'train': [0.9147906303405762, 0.912767231464386], 'val': [0.987281322479248, 0.8960899710655212]},
lr: 0.031449921228290434, Lambda: 3.1478848403116417e-06
# Display evaluation table
df_eval
| LR | Lambda | train_loss | train_accuracy | val_loss | val_accuracy | |
|---|---|---|---|---|---|---|
| 0 | 2.365837 | 0.009159 | 1.185705 | 0.835044 | 1.193376 | 0.833500 |
| 1 | 3.902845 | 0.000001 | 1.192948 | 0.835044 | 1.200686 | 0.833500 |
| 2 | 0.363104 | 0.000061 | 0.600174 | 0.963549 | 1.227158 | 0.880469 |
| 3 | 0.023226 | 0.000313 | 0.985064 | 0.898697 | 1.011453 | 0.890100 |
| 4 | 1.837335 | 0.000319 | 1.202507 | 0.835044 | 1.210242 | 0.833500 |
| ... | ... | ... | ... | ... | ... | ... |
| 85 | 0.011786 | 0.000003 | 0.989465 | 0.898216 | 1.040352 | 0.882421 |
| 86 | 0.063965 | 0.000003 | 0.855303 | 0.923570 | 0.999191 | 0.898499 |
| 87 | 0.015296 | 0.000003 | 0.962583 | 0.903070 | 1.013038 | 0.891631 |
| 88 | 0.051082 | 0.000003 | 0.886023 | 0.919236 | 0.982415 | 0.898697 |
| 89 | 0.031450 | 0.000003 | 0.914791 | 0.912767 | 0.987281 | 0.896090 |
90 rows × 6 columns
from mpl_toolkits import mplot3d
import numpy as np
import matplotlib.pyplot as plt
plt.figure(figsize = (30, 30))
# all three axes
x = df_eval["LR"]
y = df_eval["Lambda"]
z = df_eval["val_accuracy"]
ax = plt.axes(projection ='3d')
ax.scatter(x, y, z, c = z, cmap ='viridis', linewidth = 0.25);
ax = plt.axes(projection ='3d')
ax.plot_trisurf(x, y, z, cmap ='viridis', edgecolor ='green');
ax.set_xlabel("x")
Text(0.5, 0, 'x')
iterations = 100
learning_rate = 0.06372093096397173
Lambda = 3.1478848403116417e-06
hidden_node1 = 26
hidden_node2 = 14
output_nodes = 6
model = Sequential()
model.add(Dense(hidden_node1, input_shape=(11,), activation='relu'))
model.add(Dense(hidden_node2, activation='relu'))
model.add(Dense(output_nodes, activation='softmax', kernel_regularizer=regularizers.l2(Lambda)))
sgd = optimizers.SGD(lr=learning_rate, decay=1e-6, momentum=0.9)
# Compile model
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['AUC'])
# Fit the model
hist = model.fit(np.array(X_train), y_train, epochs=iterations, batch_size=1000, verbose=2,
validation_data=(X_val, y_val))
Epoch 1/100 1/1 - 0s - loss: 1.8186 - auc: 0.6023 - val_loss: 1.7572 - val_auc: 0.6441 Epoch 2/100 1/1 - 0s - loss: 1.7610 - auc: 0.6373 - val_loss: 1.6693 - val_auc: 0.6980 Epoch 3/100 1/1 - 0s - loss: 1.6675 - auc: 0.6968 - val_loss: 1.5715 - val_auc: 0.7595 Epoch 4/100 1/1 - 0s - loss: 1.5633 - auc: 0.7623 - val_loss: 1.4797 - val_auc: 0.8062 Epoch 5/100 1/1 - 0s - loss: 1.4664 - auc: 0.8093 - val_loss: 1.4015 - val_auc: 0.8329 Epoch 6/100 1/1 - 0s - loss: 1.3836 - auc: 0.8366 - val_loss: 1.3368 - val_auc: 0.8439 Epoch 7/100 1/1 - 0s - loss: 1.3150 - auc: 0.8501 - val_loss: 1.2833 - val_auc: 0.8486 Epoch 8/100 1/1 - 0s - loss: 1.2585 - auc: 0.8569 - val_loss: 1.2406 - val_auc: 0.8525 Epoch 9/100 1/1 - 0s - loss: 1.2129 - auc: 0.8614 - val_loss: 1.2094 - val_auc: 0.8554 Epoch 10/100 1/1 - 0s - loss: 1.1775 - auc: 0.8652 - val_loss: 1.1877 - val_auc: 0.8583 Epoch 11/100 1/1 - 0s - loss: 1.1518 - auc: 0.8682 - val_loss: 1.1730 - val_auc: 0.8606 Epoch 12/100 1/1 - 0s - loss: 1.1339 - auc: 0.8709 - val_loss: 1.1620 - val_auc: 0.8635 Epoch 13/100 1/1 - 0s - loss: 1.1210 - auc: 0.8732 - val_loss: 1.1524 - val_auc: 0.8662 Epoch 14/100 1/1 - 0s - loss: 1.1104 - auc: 0.8759 - val_loss: 1.1431 - val_auc: 0.8689 Epoch 15/100 1/1 - 0s - loss: 1.1008 - auc: 0.8784 - val_loss: 1.1342 - val_auc: 0.8714 Epoch 16/100 1/1 - 0s - loss: 1.0921 - auc: 0.8808 - val_loss: 1.1264 - val_auc: 0.8738 Epoch 17/100 1/1 - 0s - loss: 1.0850 - auc: 0.8822 - val_loss: 1.1196 - val_auc: 0.8742 Epoch 18/100 1/1 - 0s - loss: 1.0795 - auc: 0.8833 - val_loss: 1.1129 - val_auc: 0.8753 Epoch 19/100 1/1 - 0s - loss: 1.0742 - auc: 0.8846 - val_loss: 1.1053 - val_auc: 0.8777 Epoch 20/100 1/1 - 0s - loss: 1.0674 - auc: 0.8860 - val_loss: 1.0965 - val_auc: 0.8798 Epoch 21/100 1/1 - 0s - loss: 1.0586 - auc: 0.8874 - val_loss: 1.0874 - val_auc: 0.8815 Epoch 22/100 1/1 - 0s - loss: 1.0485 - auc: 0.8891 - val_loss: 1.0792 - val_auc: 0.8828 Epoch 23/100 1/1 - 0s - loss: 1.0387 - auc: 0.8910 - val_loss: 1.0720 - val_auc: 0.8835 Epoch 24/100 1/1 - 0s - loss: 1.0295 - auc: 0.8925 - val_loss: 1.0657 - val_auc: 0.8847 Epoch 25/100 1/1 - 0s - loss: 1.0211 - auc: 0.8934 - val_loss: 1.0597 - val_auc: 0.8853 Epoch 26/100 1/1 - 0s - loss: 1.0131 - auc: 0.8949 - val_loss: 1.0539 - val_auc: 0.8864 Epoch 27/100 1/1 - 0s - loss: 1.0054 - auc: 0.8963 - val_loss: 1.0482 - val_auc: 0.8865 Epoch 28/100 1/1 - 0s - loss: 0.9980 - auc: 0.8970 - val_loss: 1.0429 - val_auc: 0.8874 Epoch 29/100 1/1 - 0s - loss: 0.9910 - auc: 0.8987 - val_loss: 1.0377 - val_auc: 0.8883 Epoch 30/100 1/1 - 0s - loss: 0.9845 - auc: 0.8995 - val_loss: 1.0329 - val_auc: 0.8883 Epoch 31/100 1/1 - 0s - loss: 0.9786 - auc: 0.9008 - val_loss: 1.0284 - val_auc: 0.8898 Epoch 32/100 1/1 - 0s - loss: 0.9734 - auc: 0.9021 - val_loss: 1.0244 - val_auc: 0.8902 Epoch 33/100 1/1 - 0s - loss: 0.9689 - auc: 0.9024 - val_loss: 1.0213 - val_auc: 0.8907 Epoch 34/100 1/1 - 0s - loss: 0.9652 - auc: 0.9033 - val_loss: 1.0188 - val_auc: 0.8908 Epoch 35/100 1/1 - 0s - loss: 0.9619 - auc: 0.9039 - val_loss: 1.0170 - val_auc: 0.8909 Epoch 36/100 1/1 - 0s - loss: 0.9590 - auc: 0.9045 - val_loss: 1.0157 - val_auc: 0.8913 Epoch 37/100 1/1 - 0s - loss: 0.9563 - auc: 0.9051 - val_loss: 1.0148 - val_auc: 0.8916 Epoch 38/100 1/1 - 0s - loss: 0.9537 - auc: 0.9056 - val_loss: 1.0141 - val_auc: 0.8916 Epoch 39/100 1/1 - 0s - loss: 0.9512 - auc: 0.9063 - val_loss: 1.0135 - val_auc: 0.8920 Epoch 40/100 1/1 - 0s - loss: 0.9488 - auc: 0.9068 - val_loss: 1.0131 - val_auc: 0.8922 Epoch 41/100 1/1 - 0s - loss: 0.9466 - auc: 0.9074 - val_loss: 1.0127 - val_auc: 0.8917 Epoch 42/100 1/1 - 0s - loss: 0.9444 - auc: 0.9076 - val_loss: 1.0123 - val_auc: 0.8920 Epoch 43/100 1/1 - 0s - loss: 0.9423 - auc: 0.9080 - val_loss: 1.0118 - val_auc: 0.8921 Epoch 44/100 1/1 - 0s - loss: 0.9399 - auc: 0.9084 - val_loss: 1.0109 - val_auc: 0.8923 Epoch 45/100 1/1 - 0s - loss: 0.9374 - auc: 0.9088 - val_loss: 1.0098 - val_auc: 0.8928 Epoch 46/100 1/1 - 0s - loss: 0.9348 - auc: 0.9094 - val_loss: 1.0087 - val_auc: 0.8929 Epoch 47/100 1/1 - 0s - loss: 0.9322 - auc: 0.9097 - val_loss: 1.0076 - val_auc: 0.8929 Epoch 48/100 1/1 - 0s - loss: 0.9296 - auc: 0.9105 - val_loss: 1.0065 - val_auc: 0.8934 Epoch 49/100 1/1 - 0s - loss: 0.9271 - auc: 0.9108 - val_loss: 1.0057 - val_auc: 0.8935 Epoch 50/100 1/1 - 0s - loss: 0.9248 - auc: 0.9113 - val_loss: 1.0050 - val_auc: 0.8937 Epoch 51/100 1/1 - 0s - loss: 0.9226 - auc: 0.9115 - val_loss: 1.0046 - val_auc: 0.8936 Epoch 52/100 1/1 - 0s - loss: 0.9205 - auc: 0.9119 - val_loss: 1.0043 - val_auc: 0.8938 Epoch 53/100 1/1 - 0s - loss: 0.9184 - auc: 0.9123 - val_loss: 1.0043 - val_auc: 0.8940 Epoch 54/100 1/1 - 0s - loss: 0.9163 - auc: 0.9128 - val_loss: 1.0044 - val_auc: 0.8942 Epoch 55/100 1/1 - 0s - loss: 0.9143 - auc: 0.9132 - val_loss: 1.0047 - val_auc: 0.8941 Epoch 56/100 1/1 - 0s - loss: 0.9123 - auc: 0.9135 - val_loss: 1.0051 - val_auc: 0.8943 Epoch 57/100 1/1 - 0s - loss: 0.9104 - auc: 0.9139 - val_loss: 1.0056 - val_auc: 0.8938 Epoch 58/100 1/1 - 0s - loss: 0.9085 - auc: 0.9143 - val_loss: 1.0059 - val_auc: 0.8939 Epoch 59/100 1/1 - 0s - loss: 0.9067 - auc: 0.9147 - val_loss: 1.0061 - val_auc: 0.8938 Epoch 60/100 1/1 - 0s - loss: 0.9048 - auc: 0.9151 - val_loss: 1.0062 - val_auc: 0.8938 Epoch 61/100 1/1 - 0s - loss: 0.9030 - auc: 0.9155 - val_loss: 1.0061 - val_auc: 0.8940 Epoch 62/100 1/1 - 0s - loss: 0.9012 - auc: 0.9158 - val_loss: 1.0060 - val_auc: 0.8943 Epoch 63/100 1/1 - 0s - loss: 0.8994 - auc: 0.9160 - val_loss: 1.0058 - val_auc: 0.8943 Epoch 64/100 1/1 - 0s - loss: 0.8977 - auc: 0.9163 - val_loss: 1.0056 - val_auc: 0.8943 Epoch 65/100 1/1 - 0s - loss: 0.8959 - auc: 0.9166 - val_loss: 1.0053 - val_auc: 0.8944 Epoch 66/100 1/1 - 0s - loss: 0.8942 - auc: 0.9170 - val_loss: 1.0050 - val_auc: 0.8950 Epoch 67/100 1/1 - 0s - loss: 0.8925 - auc: 0.9173 - val_loss: 1.0046 - val_auc: 0.8951 Epoch 68/100 1/1 - 0s - loss: 0.8909 - auc: 0.9176 - val_loss: 1.0043 - val_auc: 0.8952 Epoch 69/100 1/1 - 0s - loss: 0.8892 - auc: 0.9180 - val_loss: 1.0039 - val_auc: 0.8954 Epoch 70/100 1/1 - 0s - loss: 0.8876 - auc: 0.9183 - val_loss: 1.0034 - val_auc: 0.8955 Epoch 71/100 1/1 - 0s - loss: 0.8860 - auc: 0.9185 - val_loss: 1.0030 - val_auc: 0.8957 Epoch 72/100 1/1 - 0s - loss: 0.8843 - auc: 0.9188 - val_loss: 1.0026 - val_auc: 0.8959 Epoch 73/100 1/1 - 0s - loss: 0.8828 - auc: 0.9191 - val_loss: 1.0021 - val_auc: 0.8960 Epoch 74/100 1/1 - 0s - loss: 0.8812 - auc: 0.9193 - val_loss: 1.0015 - val_auc: 0.8961 Epoch 75/100 1/1 - 0s - loss: 0.8796 - auc: 0.9195 - val_loss: 1.0009 - val_auc: 0.8962 Epoch 76/100 1/1 - 0s - loss: 0.8781 - auc: 0.9198 - val_loss: 1.0004 - val_auc: 0.8963 Epoch 77/100 1/1 - 0s - loss: 0.8765 - auc: 0.9200 - val_loss: 0.9998 - val_auc: 0.8965 Epoch 78/100 1/1 - 0s - loss: 0.8749 - auc: 0.9204 - val_loss: 0.9992 - val_auc: 0.8968 Epoch 79/100 1/1 - 0s - loss: 0.8734 - auc: 0.9208 - val_loss: 0.9988 - val_auc: 0.8968 Epoch 80/100 1/1 - 0s - loss: 0.8718 - auc: 0.9211 - val_loss: 0.9984 - val_auc: 0.8969 Epoch 81/100 1/1 - 0s - loss: 0.8703 - auc: 0.9214 - val_loss: 0.9981 - val_auc: 0.8971 Epoch 82/100 1/1 - 0s - loss: 0.8688 - auc: 0.9216 - val_loss: 0.9979 - val_auc: 0.8971 Epoch 83/100 1/1 - 0s - loss: 0.8674 - auc: 0.9219 - val_loss: 0.9978 - val_auc: 0.8971 Epoch 84/100 1/1 - 0s - loss: 0.8659 - auc: 0.9221 - val_loss: 0.9976 - val_auc: 0.8972 Epoch 85/100 1/1 - 0s - loss: 0.8645 - auc: 0.9223 - val_loss: 0.9975 - val_auc: 0.8972 Epoch 86/100 1/1 - 0s - loss: 0.8631 - auc: 0.9226 - val_loss: 0.9975 - val_auc: 0.8970 Epoch 87/100 1/1 - 0s - loss: 0.8617 - auc: 0.9228 - val_loss: 0.9975 - val_auc: 0.8970 Epoch 88/100 1/1 - 0s - loss: 0.8603 - auc: 0.9230 - val_loss: 0.9975 - val_auc: 0.8969 Epoch 89/100 1/1 - 0s - loss: 0.8590 - auc: 0.9232 - val_loss: 0.9975 - val_auc: 0.8969 Epoch 90/100 1/1 - 0s - loss: 0.8576 - auc: 0.9235 - val_loss: 0.9976 - val_auc: 0.8970 Epoch 91/100 1/1 - 0s - loss: 0.8562 - auc: 0.9237 - val_loss: 0.9977 - val_auc: 0.8970 Epoch 92/100 1/1 - 0s - loss: 0.8549 - auc: 0.9240 - val_loss: 0.9978 - val_auc: 0.8970 Epoch 93/100 1/1 - 0s - loss: 0.8535 - auc: 0.9242 - val_loss: 0.9979 - val_auc: 0.8971 Epoch 94/100 1/1 - 0s - loss: 0.8523 - auc: 0.9245 - val_loss: 0.9979 - val_auc: 0.8971 Epoch 95/100 1/1 - 0s - loss: 0.8510 - auc: 0.9246 - val_loss: 0.9978 - val_auc: 0.8972 Epoch 96/100 1/1 - 0s - loss: 0.8497 - auc: 0.9248 - val_loss: 0.9978 - val_auc: 0.8973 Epoch 97/100 1/1 - 0s - loss: 0.8484 - auc: 0.9251 - val_loss: 0.9977 - val_auc: 0.8974 Epoch 98/100 1/1 - 0s - loss: 0.8471 - auc: 0.9253 - val_loss: 0.9976 - val_auc: 0.8976 Epoch 99/100 1/1 - 0s - loss: 0.8458 - auc: 0.9254 - val_loss: 0.9975 - val_auc: 0.8977 Epoch 100/100 1/1 - 0s - loss: 0.8445 - auc: 0.9256 - val_loss: 0.9974 - val_auc: 0.8977
# Evaluation of the model against validation set separately
model.evaluate(np.array(X_val), y_val, verbose=1)
13/13 [==============================] - 0s 2ms/step - loss: 0.9974 - auc: 0.8977
[0.997394859790802, 0.8977106213569641]
# Running the final model on the test set
model.evaluate(np.array(X_test), np.array(y_test), verbose=0)
[0.9071255326271057, 0.914591908454895]
import pickle
from keras_pickle_wrapper import KerasPickleWrapper
# Wrap a compiled model
mw = KerasPickleWrapper(model)
# Create Pickle file from the pipeline created around the final model
with open('final_model.pickle', 'wb') as dump_var:
pickle.dump(mw, dump_var)
DOMAIN: Autonomous Vehicles
BUSINESS CONTEXT: A Recognising multi-digit numbers in photographs captured at street level is an important component of modern-day map making. A classic example of a corpus of such street-level photographs is Google’s Street View imagery composed of hundreds of millions of geo-located 360-degree panoramic images. The ability to automatically transcribe an address number from a geo-located patch of pixels and associate the transcribed number with a known street address helps pinpoint, with a high degree of accuracy, the location of the building it represents. More broadly, recognising numbers in photographs is a problem of interest to the optical character recognition community. While OCR on constrained domains like document processing is well studied, arbitrary multi-character text recognition in photographs is still highly challenging. This difficulty arises due to the wide variability in the visual appearance of text in the wild on account of a large range of fonts, colours, styles, orientations, and character arrangements. The recognition problem is further complicated by environmental factors such as lighting, shadows, specularity, and occlusions as well as by image acquisition factors such as resolution, motion, and focus blurs. In this project, we will use the dataset with images centred around a single digit (many of the images do contain some distractors at the sides). Although we are taking a sample of the data which is simpler, it is more complex than MNIST because of the distractors.
DATA DESCRIPTION: The SVHN is a real-world image dataset for developing machine learning and object recognition algorithms with the minimal requirement on data formatting but comes from a significantly harder, unsolved, real-world problem (recognising digits and numbers in natural scene images). SVHN is obtained from house numbers in Google Street View images. Where the labels for each of this image are the prominent number in that image i.e. 2,6,7 and 4 respectively.The dataset has been provided in the form of h5py files.
PROJECT OBJECTIVE: We will build a digit classifier on the SVHN (Street View Housing Number) dataset.
Steps and tasks:
import pandas as pd
import numpy as np
import h5py
import tensorflow
%matplotlib inline
import matplotlib.pyplot as plt
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras import regularizers, optimizers
import math
# Import Data
h5 = h5py.File('Part - 4 - Autonomous_Vehicles_SVHN_single_grey1.h5','r')
# Check keys
h5.keys()
<KeysViewHDF5 ['X_test', 'X_train', 'X_val', 'y_test', 'y_train', 'y_val']>
# Load data into train, validation and test set
X_train = np.array(h5['X_val'])
X_test = np.array(h5['X_test'])
X_val = np.array(h5['X_train'])
y_train = np.array(h5['y_val'])
y_test = np.array(h5['y_test'])
y_val = np.array(h5['y_train'])
print("Train")
print(X_train.shape)
print(y_train.shape)
print("Validation")
print(X_val.shape)
print(y_val.shape)
print("Test")
print(X_test.shape)
print(y_test.shape)
Train (60000, 32, 32) (60000,) Validation (42000, 32, 32) (42000,) Test (18000, 32, 32) (18000,)
# Reshape data
X_train = X_train.reshape(60000, 1024)
print(X_train.shape)
X_val = X_val.reshape(42000, 1024)
print(X_val.shape)
X_test = X_test.reshape(18000, 1024)
print(X_test.shape)
(60000, 1024) (42000, 1024) (18000, 1024)
# Standardise Data
print("Range", X_train.max() - X_train.min())
X_train = X_train / 255.0
X_val = X_val / 255.0
print(X_train.max())
print(X_train.min())
Range 254.9745 0.9999 0.0
# Rearrange target variable for NN classifier
print(y_train[10])
y_train = tensorflow.keras.utils.to_categorical(y_train, num_classes=10)
y_val = tensorflow.keras.utils.to_categorical(y_val, num_classes=10)
y_test = tensorflow.keras.utils.to_categorical(y_test, num_classes=10)
print(y_train[10])
0 [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
# visualizing the first 10 images in the dataset and their labels
plt.figure(figsize=(10, 1))
for i in range(10):
plt.subplot(1, 10, i+1)
plt.imshow(X_train[i].reshape(32, 32), cmap="gray")
plt.axis('off')
print('label for each of the below image: %s' % (np.argmax(y_train[0:10][i])))
plt.show()
label for each of the below image: 0 label for each of the below image: 0 label for each of the below image: 0 label for each of the below image: 0 label for each of the below image: 0 label for each of the below image: 0 label for each of the below image: 0 label for each of the below image: 0 label for each of the below image: 0 label for each of the below image: 0
# Viewing specific images
plt.imshow(np.array(h5['X_train'])[4])
np.array(h5['y_train'])[4]
4
# Checking the distribution of the results
for i in range(y_val.shape[1]):
cnt = 0
for j in range(y_val.shape[0]):
cnt += y_val[j][i]
print(i, cnt)
0 4186.0 1 4172.0 2 4197.0 3 4281.0 4 4188.0 5 4232.0 6 4168.0 7 4192.0 8 4188.0 9 4196.0
Hint: Use best approach to refine and tune the data or the model. Be highly experimental here to get the best accuracy out of the model
def train_and_test_loop(iterations, lr, Lambda, verb=True):
## hyperparameters
iterations = iterations
learning_rate = lr
hidden_nodes = 339
output_nodes = 10
model = Sequential()
model.add(Dense(hidden_nodes, input_shape=(1024,), activation='relu'))
model.add(Dense(hidden_nodes, activation='relu'))
model.add(Dense(output_nodes, activation='softmax', kernel_regularizer=regularizers.l2(Lambda)))
sgd = optimizers.SGD(lr=learning_rate, decay=1e-6, momentum=0.9)
# Compile model
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
# Fit the model
model.fit(X_train, y_train, epochs=iterations, batch_size=1000, verbose= 1)
def train_and_test_loop1(iterations, lr, Lambda, verb=True):
## hyperparameters
iterations = iterations
learning_rate = lr
hidden_nodes = 339
output_nodes = 10
model = Sequential()
model.add(Dense(hidden_nodes, input_shape=(1024,), activation='relu'))
model.add(Dense(hidden_nodes, activation='relu'))
model.add(Dense(output_nodes, activation='softmax', kernel_regularizer=regularizers.l2(Lambda)))
sgd = optimizers.SGD(lr=learning_rate, decay=1e-6, momentum=0.9)
# Compile model
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
# Fit the model
model.fit(X_train, y_train, epochs=iterations, batch_size=1000, verbose= 0)
score = model.evaluate(X_train, y_train, verbose=0)
return score
lr = 0.00001
Lambda = 0
train_and_test_loop(1, lr, Lambda)
42/42 [==============================] - 1s 19ms/step - loss: 2.3612 - accuracy: 0.0999
lr = 0.00001
Lambda = 1e3
train_and_test_loop(1, lr, Lambda)
42/42 [==============================] - 1s 19ms/step - loss: 3335.3435 - accuracy: 0.1010
lr = 0.001
Lambda = 0
train_and_test_loop(500, lr, Lambda)
Epoch 1/500 1/1 [==============================] - 0s 0s/step - loss: 2.3780 - accuracy: 0.0000e+00 Epoch 2/500 1/1 [==============================] - 0s 1ms/step - loss: 2.3554 - accuracy: 0.0000e+00 Epoch 3/500 1/1 [==============================] - 0s 1ms/step - loss: 2.3144 - accuracy: 0.0000e+00 Epoch 4/500 1/1 [==============================] - 0s 1ms/step - loss: 2.2603 - accuracy: 0.0000e+00 Epoch 5/500 1/1 [==============================] - 0s 998us/step - loss: 2.1986 - accuracy: 0.1500 Epoch 6/500 1/1 [==============================] - 0s 997us/step - loss: 2.1341 - accuracy: 0.2000 Epoch 7/500 1/1 [==============================] - 0s 997us/step - loss: 2.0731 - accuracy: 0.2000 Epoch 8/500 1/1 [==============================] - 0s 2ms/step - loss: 2.0189 - accuracy: 0.2500 Epoch 9/500 1/1 [==============================] - 0s 2ms/step - loss: 1.9734 - accuracy: 0.2500 Epoch 10/500 1/1 [==============================] - 0s 993us/step - loss: 1.9372 - accuracy: 0.3000 Epoch 11/500 1/1 [==============================] - 0s 1ms/step - loss: 1.9102 - accuracy: 0.4000 Epoch 12/500 1/1 [==============================] - 0s 1000us/step - loss: 1.8898 - accuracy: 0.3500 Epoch 13/500 1/1 [==============================] - 0s 2ms/step - loss: 1.8752 - accuracy: 0.3500 Epoch 14/500 1/1 [==============================] - 0s 2ms/step - loss: 1.8641 - accuracy: 0.3500 Epoch 15/500 1/1 [==============================] - 0s 2ms/step - loss: 1.8542 - accuracy: 0.3500 Epoch 16/500 1/1 [==============================] - 0s 1ms/step - loss: 1.8450 - accuracy: 0.3500 Epoch 17/500 1/1 [==============================] - 0s 995us/step - loss: 1.8355 - accuracy: 0.3500 Epoch 18/500 1/1 [==============================] - 0s 995us/step - loss: 1.8249 - accuracy: 0.3500 Epoch 19/500 1/1 [==============================] - 0s 1ms/step - loss: 1.8135 - accuracy: 0.3500 Epoch 20/500 1/1 [==============================] - 0s 1ms/step - loss: 1.8016 - accuracy: 0.3500 Epoch 21/500 1/1 [==============================] - 0s 2ms/step - loss: 1.7893 - accuracy: 0.3500 Epoch 22/500 1/1 [==============================] - 0s 1ms/step - loss: 1.7768 - accuracy: 0.4000 Epoch 23/500 1/1 [==============================] - 0s 2ms/step - loss: 1.7645 - accuracy: 0.4000 Epoch 24/500 1/1 [==============================] - 0s 998us/step - loss: 1.7527 - accuracy: 0.4000 Epoch 25/500 1/1 [==============================] - 0s 2ms/step - loss: 1.7417 - accuracy: 0.5500 Epoch 26/500 1/1 [==============================] - 0s 2ms/step - loss: 1.7311 - accuracy: 0.5500 Epoch 27/500 1/1 [==============================] - 0s 998us/step - loss: 1.7210 - accuracy: 0.5000 Epoch 28/500 1/1 [==============================] - 0s 2ms/step - loss: 1.7112 - accuracy: 0.5000 Epoch 29/500 1/1 [==============================] - 0s 2ms/step - loss: 1.7017 - accuracy: 0.5000 Epoch 30/500 1/1 [==============================] - 0s 2ms/step - loss: 1.6926 - accuracy: 0.5000 Epoch 31/500 1/1 [==============================] - 0s 998us/step - loss: 1.6837 - accuracy: 0.5000 Epoch 32/500 1/1 [==============================] - 0s 2ms/step - loss: 1.6751 - accuracy: 0.5000 Epoch 33/500 1/1 [==============================] - 0s 997us/step - loss: 1.6669 - accuracy: 0.5000 Epoch 34/500 1/1 [==============================] - 0s 995us/step - loss: 1.6592 - accuracy: 0.5500 Epoch 35/500 1/1 [==============================] - 0s 998us/step - loss: 1.6518 - accuracy: 0.5500 Epoch 36/500 1/1 [==============================] - 0s 2ms/step - loss: 1.6445 - accuracy: 0.5000 Epoch 37/500 1/1 [==============================] - 0s 2ms/step - loss: 1.6372 - accuracy: 0.5000 Epoch 38/500 1/1 [==============================] - 0s 2ms/step - loss: 1.6299 - accuracy: 0.5000 Epoch 39/500 1/1 [==============================] - 0s 2ms/step - loss: 1.6225 - accuracy: 0.5000 Epoch 40/500 1/1 [==============================] - 0s 998us/step - loss: 1.6154 - accuracy: 0.5000 Epoch 41/500 1/1 [==============================] - 0s 997us/step - loss: 1.6085 - accuracy: 0.5000 Epoch 42/500 1/1 [==============================] - 0s 2ms/step - loss: 1.6015 - accuracy: 0.5000 Epoch 43/500 1/1 [==============================] - 0s 997us/step - loss: 1.5944 - accuracy: 0.5000 Epoch 44/500 1/1 [==============================] - 0s 2ms/step - loss: 1.5873 - accuracy: 0.5000 Epoch 45/500 1/1 [==============================] - 0s 1ms/step - loss: 1.5802 - accuracy: 0.5000 Epoch 46/500 1/1 [==============================] - 0s 2ms/step - loss: 1.5733 - accuracy: 0.5500 Epoch 47/500 1/1 [==============================] - 0s 2ms/step - loss: 1.5664 - accuracy: 0.5500 Epoch 48/500 1/1 [==============================] - 0s 2ms/step - loss: 1.5595 - accuracy: 0.5500 Epoch 49/500 1/1 [==============================] - 0s 995us/step - loss: 1.5530 - accuracy: 0.5500 Epoch 50/500 1/1 [==============================] - 0s 2ms/step - loss: 1.5465 - accuracy: 0.5500 Epoch 51/500 1/1 [==============================] - 0s 999us/step - loss: 1.5400 - accuracy: 0.5500 Epoch 52/500 1/1 [==============================] - 0s 998us/step - loss: 1.5337 - accuracy: 0.5500 Epoch 53/500 1/1 [==============================] - 0s 998us/step - loss: 1.5275 - accuracy: 0.5500 Epoch 54/500 1/1 [==============================] - 0s 2ms/step - loss: 1.5211 - accuracy: 0.5500 Epoch 55/500 1/1 [==============================] - 0s 998us/step - loss: 1.5148 - accuracy: 0.5500 Epoch 56/500 1/1 [==============================] - 0s 2ms/step - loss: 1.5086 - accuracy: 0.5500 Epoch 57/500 1/1 [==============================] - 0s 999us/step - loss: 1.5025 - accuracy: 0.5500 Epoch 58/500 1/1 [==============================] - 0s 956us/step - loss: 1.4963 - accuracy: 0.5500 Epoch 59/500 1/1 [==============================] - 0s 2ms/step - loss: 1.4900 - accuracy: 0.5500 Epoch 60/500 1/1 [==============================] - 0s 2ms/step - loss: 1.4838 - accuracy: 0.5500 Epoch 61/500 1/1 [==============================] - 0s 996us/step - loss: 1.4776 - accuracy: 0.5500 Epoch 62/500 1/1 [==============================] - 0s 3ms/step - loss: 1.4714 - accuracy: 0.5500 Epoch 63/500 1/1 [==============================] - 0s 997us/step - loss: 1.4652 - accuracy: 0.5500 Epoch 64/500 1/1 [==============================] - 0s 997us/step - loss: 1.4590 - accuracy: 0.5500 Epoch 65/500 1/1 [==============================] - 0s 2ms/step - loss: 1.4527 - accuracy: 0.5500 Epoch 66/500 1/1 [==============================] - 0s 0s/step - loss: 1.4466 - accuracy: 0.5500 Epoch 67/500 1/1 [==============================] - 0s 0s/step - loss: 1.4406 - accuracy: 0.5500 Epoch 68/500 1/1 [==============================] - 0s 1ms/step - loss: 1.4346 - accuracy: 0.5500 Epoch 69/500 1/1 [==============================] - 0s 1000us/step - loss: 1.4286 - accuracy: 0.5500 Epoch 70/500 1/1 [==============================] - 0s 997us/step - loss: 1.4227 - accuracy: 0.5500 Epoch 71/500 1/1 [==============================] - 0s 997us/step - loss: 1.4169 - accuracy: 0.5500 Epoch 72/500 1/1 [==============================] - 0s 996us/step - loss: 1.4110 - accuracy: 0.5500 Epoch 73/500 1/1 [==============================] - 0s 996us/step - loss: 1.4051 - accuracy: 0.5500 Epoch 74/500 1/1 [==============================] - 0s 1ms/step - loss: 1.3992 - accuracy: 0.5500 Epoch 75/500 1/1 [==============================] - 0s 997us/step - loss: 1.3933 - accuracy: 0.5500 Epoch 76/500 1/1 [==============================] - 0s 1ms/step - loss: 1.3873 - accuracy: 0.5500 Epoch 77/500 1/1 [==============================] - 0s 3ms/step - loss: 1.3814 - accuracy: 0.5500 Epoch 78/500 1/1 [==============================] - 0s 997us/step - loss: 1.3757 - accuracy: 0.5500 Epoch 79/500 1/1 [==============================] - 0s 998us/step - loss: 1.3700 - accuracy: 0.5500 Epoch 80/500 1/1 [==============================] - 0s 1ms/step - loss: 1.3642 - accuracy: 0.5500 Epoch 81/500 1/1 [==============================] - 0s 1ms/step - loss: 1.3585 - accuracy: 0.5500 Epoch 82/500 1/1 [==============================] - 0s 999us/step - loss: 1.3528 - accuracy: 0.5500 Epoch 83/500 1/1 [==============================] - 0s 996us/step - loss: 1.3471 - accuracy: 0.5500 Epoch 84/500 1/1 [==============================] - 0s 998us/step - loss: 1.3413 - accuracy: 0.5500 Epoch 85/500 1/1 [==============================] - 0s 1ms/step - loss: 1.3356 - accuracy: 0.5500 Epoch 86/500 1/1 [==============================] - 0s 1ms/step - loss: 1.3299 - accuracy: 0.5500 Epoch 87/500 1/1 [==============================] - 0s 2ms/step - loss: 1.3242 - accuracy: 0.5500 Epoch 88/500 1/1 [==============================] - 0s 2ms/step - loss: 1.3186 - accuracy: 0.5500 Epoch 89/500 1/1 [==============================] - 0s 1ms/step - loss: 1.3129 - accuracy: 0.5500 Epoch 90/500 1/1 [==============================] - 0s 996us/step - loss: 1.3073 - accuracy: 0.5500 Epoch 91/500 1/1 [==============================] - 0s 3ms/step - loss: 1.3016 - accuracy: 0.6000 Epoch 92/500 1/1 [==============================] - 0s 999us/step - loss: 1.2960 - accuracy: 0.6000 Epoch 93/500 1/1 [==============================] - 0s 2ms/step - loss: 1.2904 - accuracy: 0.6000 Epoch 94/500 1/1 [==============================] - 0s 993us/step - loss: 1.2848 - accuracy: 0.6000 Epoch 95/500 1/1 [==============================] - 0s 2ms/step - loss: 1.2793 - accuracy: 0.6000 Epoch 96/500 1/1 [==============================] - 0s 1ms/step - loss: 1.2737 - accuracy: 0.6500 Epoch 97/500 1/1 [==============================] - 0s 2ms/step - loss: 1.2682 - accuracy: 0.7000 Epoch 98/500 1/1 [==============================] - 0s 997us/step - loss: 1.2627 - accuracy: 0.7000 Epoch 99/500 1/1 [==============================] - 0s 1ms/step - loss: 1.2572 - accuracy: 0.7000 Epoch 100/500 1/1 [==============================] - 0s 2ms/step - loss: 1.2517 - accuracy: 0.7000 Epoch 101/500 1/1 [==============================] - 0s 997us/step - loss: 1.2463 - accuracy: 0.7000 Epoch 102/500 1/1 [==============================] - 0s 1ms/step - loss: 1.2408 - accuracy: 0.7000 Epoch 103/500 1/1 [==============================] - 0s 997us/step - loss: 1.2354 - accuracy: 0.7000 Epoch 104/500 1/1 [==============================] - 0s 1ms/step - loss: 1.2299 - accuracy: 0.7000 Epoch 105/500 1/1 [==============================] - 0s 2ms/step - loss: 1.2244 - accuracy: 0.7000 Epoch 106/500 1/1 [==============================] - 0s 1ms/step - loss: 1.2190 - accuracy: 0.7000 Epoch 107/500 1/1 [==============================] - 0s 2ms/step - loss: 1.2135 - accuracy: 0.7500 Epoch 108/500 1/1 [==============================] - 0s 997us/step - loss: 1.2081 - accuracy: 0.7500 Epoch 109/500 1/1 [==============================] - 0s 995us/step - loss: 1.2026 - accuracy: 0.7500 Epoch 110/500 1/1 [==============================] - 0s 996us/step - loss: 1.1972 - accuracy: 0.7500 Epoch 111/500 1/1 [==============================] - 0s 996us/step - loss: 1.1919 - accuracy: 0.7500 Epoch 112/500 1/1 [==============================] - 0s 1ms/step - loss: 1.1865 - accuracy: 0.7500 Epoch 113/500 1/1 [==============================] - 0s 998us/step - loss: 1.1811 - accuracy: 0.7500 Epoch 114/500 1/1 [==============================] - 0s 998us/step - loss: 1.1757 - accuracy: 0.7500 Epoch 115/500 1/1 [==============================] - 0s 0s/step - loss: 1.1703 - accuracy: 0.7500 Epoch 116/500 1/1 [==============================] - 0s 1ms/step - loss: 1.1650 - accuracy: 0.7500 Epoch 117/500 1/1 [==============================] - 0s 996us/step - loss: 1.1598 - accuracy: 0.7500 Epoch 118/500 1/1 [==============================] - 0s 1ms/step - loss: 1.1546 - accuracy: 0.7500 Epoch 119/500 1/1 [==============================] - 0s 2ms/step - loss: 1.1495 - accuracy: 0.7500 Epoch 120/500 1/1 [==============================] - 0s 266us/step - loss: 1.1443 - accuracy: 0.7500 Epoch 121/500 1/1 [==============================] - 0s 995us/step - loss: 1.1392 - accuracy: 0.7500 Epoch 122/500 1/1 [==============================] - 0s 999us/step - loss: 1.1341 - accuracy: 0.7500 Epoch 123/500 1/1 [==============================] - 0s 1ms/step - loss: 1.1291 - accuracy: 0.7500 Epoch 124/500 1/1 [==============================] - 0s 997us/step - loss: 1.1240 - accuracy: 0.7500 Epoch 125/500 1/1 [==============================] - 0s 995us/step - loss: 1.1189 - accuracy: 0.7500 Epoch 126/500 1/1 [==============================] - 0s 1ms/step - loss: 1.1138 - accuracy: 0.7500 Epoch 127/500 1/1 [==============================] - 0s 989us/step - loss: 1.1087 - accuracy: 0.7500 Epoch 128/500 1/1 [==============================] - 0s 3ms/step - loss: 1.1037 - accuracy: 0.7500 Epoch 129/500 1/1 [==============================] - 0s 996us/step - loss: 1.0985 - accuracy: 0.7500 Epoch 130/500 1/1 [==============================] - 0s 1ms/step - loss: 1.0934 - accuracy: 0.8000 Epoch 131/500 1/1 [==============================] - 0s 2ms/step - loss: 1.0883 - accuracy: 0.8000 Epoch 132/500 1/1 [==============================] - 0s 996us/step - loss: 1.0832 - accuracy: 0.8000 Epoch 133/500 1/1 [==============================] - 0s 998us/step - loss: 1.0781 - accuracy: 0.8000 Epoch 134/500 1/1 [==============================] - 0s 997us/step - loss: 1.0730 - accuracy: 0.8000 Epoch 135/500 1/1 [==============================] - 0s 2ms/step - loss: 1.0679 - accuracy: 0.8000 Epoch 136/500 1/1 [==============================] - 0s 657us/step - loss: 1.0628 - accuracy: 0.8000 Epoch 137/500 1/1 [==============================] - 0s 1ms/step - loss: 1.0576 - accuracy: 0.8000 Epoch 138/500 1/1 [==============================] - 0s 1ms/step - loss: 1.0525 - accuracy: 0.8000 Epoch 139/500 1/1 [==============================] - 0s 995us/step - loss: 1.0473 - accuracy: 0.8000 Epoch 140/500 1/1 [==============================] - 0s 998us/step - loss: 1.0422 - accuracy: 0.8000 Epoch 141/500 1/1 [==============================] - 0s 997us/step - loss: 1.0371 - accuracy: 0.8000 Epoch 142/500 1/1 [==============================] - 0s 999us/step - loss: 1.0320 - accuracy: 0.8000 Epoch 143/500 1/1 [==============================] - 0s 998us/step - loss: 1.0270 - accuracy: 0.8000 Epoch 144/500 1/1 [==============================] - 0s 998us/step - loss: 1.0219 - accuracy: 0.8000 Epoch 145/500 1/1 [==============================] - 0s 1ms/step - loss: 1.0169 - accuracy: 0.8000 Epoch 146/500 1/1 [==============================] - 0s 998us/step - loss: 1.0119 - accuracy: 0.8000 Epoch 147/500 1/1 [==============================] - 0s 2ms/step - loss: 1.0069 - accuracy: 0.8000 Epoch 148/500 1/1 [==============================] - 0s 1ms/step - loss: 1.0020 - accuracy: 0.8000 Epoch 149/500 1/1 [==============================] - 0s 2ms/step - loss: 0.9971 - accuracy: 0.8000 Epoch 150/500 1/1 [==============================] - 0s 2ms/step - loss: 0.9921 - accuracy: 0.8000 Epoch 151/500 1/1 [==============================] - 0s 990us/step - loss: 0.9872 - accuracy: 0.8000 Epoch 152/500 1/1 [==============================] - 0s 997us/step - loss: 0.9823 - accuracy: 0.8000 Epoch 153/500 1/1 [==============================] - 0s 998us/step - loss: 0.9774 - accuracy: 0.8000 Epoch 154/500 1/1 [==============================] - 0s 2ms/step - loss: 0.9726 - accuracy: 0.8000 Epoch 155/500 1/1 [==============================] - 0s 1ms/step - loss: 0.9678 - accuracy: 0.8000 Epoch 156/500 1/1 [==============================] - 0s 2ms/step - loss: 0.9629 - accuracy: 0.8000 Epoch 157/500 1/1 [==============================] - 0s 2ms/step - loss: 0.9581 - accuracy: 0.8000 Epoch 158/500 1/1 [==============================] - 0s 1ms/step - loss: 0.9534 - accuracy: 0.8000 Epoch 159/500 1/1 [==============================] - 0s 998us/step - loss: 0.9487 - accuracy: 0.8000 Epoch 160/500 1/1 [==============================] - 0s 1ms/step - loss: 0.9440 - accuracy: 0.8000 Epoch 161/500 1/1 [==============================] - 0s 994us/step - loss: 0.9393 - accuracy: 0.8000 Epoch 162/500 1/1 [==============================] - 0s 995us/step - loss: 0.9346 - accuracy: 0.8000 Epoch 163/500 1/1 [==============================] - 0s 2ms/step - loss: 0.9300 - accuracy: 0.8000 Epoch 164/500 1/1 [==============================] - 0s 997us/step - loss: 0.9254 - accuracy: 0.8000 Epoch 165/500 1/1 [==============================] - 0s 997us/step - loss: 0.9208 - accuracy: 0.8000 Epoch 166/500 1/1 [==============================] - 0s 999us/step - loss: 0.9162 - accuracy: 0.8000 Epoch 167/500 1/1 [==============================] - 0s 3ms/step - loss: 0.9116 - accuracy: 0.8000 Epoch 168/500 1/1 [==============================] - 0s 1ms/step - loss: 0.9071 - accuracy: 0.8000 Epoch 169/500 1/1 [==============================] - 0s 2ms/step - loss: 0.9025 - accuracy: 0.8000 Epoch 170/500 1/1 [==============================] - 0s 1ms/step - loss: 0.8980 - accuracy: 0.8000 Epoch 171/500 1/1 [==============================] - 0s 999us/step - loss: 0.8935 - accuracy: 0.8000 Epoch 172/500 1/1 [==============================] - 0s 1ms/step - loss: 0.8890 - accuracy: 0.8000 Epoch 173/500 1/1 [==============================] - 0s 1000us/step - loss: 0.8845 - accuracy: 0.8000 Epoch 174/500 1/1 [==============================] - 0s 998us/step - loss: 0.8801 - accuracy: 0.8000 Epoch 175/500 1/1 [==============================] - 0s 999us/step - loss: 0.8757 - accuracy: 0.8000 Epoch 176/500 1/1 [==============================] - 0s 2ms/step - loss: 0.8713 - accuracy: 0.8000 Epoch 177/500 1/1 [==============================] - 0s 997us/step - loss: 0.8668 - accuracy: 0.8000 Epoch 178/500 1/1 [==============================] - 0s 1ms/step - loss: 0.8625 - accuracy: 0.8000 Epoch 179/500 1/1 [==============================] - 0s 1ms/step - loss: 0.8581 - accuracy: 0.8000 Epoch 180/500 1/1 [==============================] - 0s 998us/step - loss: 0.8537 - accuracy: 0.8000 Epoch 181/500 1/1 [==============================] - 0s 2ms/step - loss: 0.8494 - accuracy: 0.8000 Epoch 182/500 1/1 [==============================] - 0s 997us/step - loss: 0.8450 - accuracy: 0.8000 Epoch 183/500 1/1 [==============================] - 0s 2ms/step - loss: 0.8407 - accuracy: 0.8000 Epoch 184/500 1/1 [==============================] - 0s 1ms/step - loss: 0.8364 - accuracy: 0.8000 Epoch 185/500 1/1 [==============================] - 0s 2ms/step - loss: 0.8321 - accuracy: 0.8000 Epoch 186/500 1/1 [==============================] - 0s 979us/step - loss: 0.8278 - accuracy: 0.8000 Epoch 187/500 1/1 [==============================] - 0s 996us/step - loss: 0.8236 - accuracy: 0.8000 Epoch 188/500 1/1 [==============================] - 0s 1ms/step - loss: 0.8193 - accuracy: 0.8500 Epoch 189/500 1/1 [==============================] - 0s 990us/step - loss: 0.8151 - accuracy: 0.8500 Epoch 190/500 1/1 [==============================] - 0s 332us/step - loss: 0.8109 - accuracy: 0.8500 Epoch 191/500 1/1 [==============================] - 0s 994us/step - loss: 0.8066 - accuracy: 0.8500 Epoch 192/500 1/1 [==============================] - 0s 2ms/step - loss: 0.8024 - accuracy: 0.9000 Epoch 193/500 1/1 [==============================] - 0s 997us/step - loss: 0.7982 - accuracy: 0.9000 Epoch 194/500 1/1 [==============================] - 0s 2ms/step - loss: 0.7941 - accuracy: 0.9000 Epoch 195/500 1/1 [==============================] - 0s 0s/step - loss: 0.7899 - accuracy: 0.9000 Epoch 196/500 1/1 [==============================] - 0s 1ms/step - loss: 0.7857 - accuracy: 0.9000 Epoch 197/500 1/1 [==============================] - 0s 995us/step - loss: 0.7816 - accuracy: 0.9000 Epoch 198/500 1/1 [==============================] - 0s 995us/step - loss: 0.7775 - accuracy: 0.9000 Epoch 199/500 1/1 [==============================] - 0s 998us/step - loss: 0.7735 - accuracy: 0.9000 Epoch 200/500 1/1 [==============================] - 0s 2ms/step - loss: 0.7694 - accuracy: 0.9000 Epoch 201/500 1/1 [==============================] - 0s 963us/step - loss: 0.7654 - accuracy: 0.9500 Epoch 202/500 1/1 [==============================] - 0s 993us/step - loss: 0.7613 - accuracy: 0.9500 Epoch 203/500 1/1 [==============================] - 0s 990us/step - loss: 0.7573 - accuracy: 0.9500 Epoch 204/500 1/1 [==============================] - 0s 996us/step - loss: 0.7533 - accuracy: 0.9500 Epoch 205/500 1/1 [==============================] - 0s 3ms/step - loss: 0.7493 - accuracy: 0.9500 Epoch 206/500 1/1 [==============================] - 0s 0s/step - loss: 0.7454 - accuracy: 0.9500 Epoch 207/500 1/1 [==============================] - 0s 1ms/step - loss: 0.7414 - accuracy: 0.9500 Epoch 208/500 1/1 [==============================] - 0s 2ms/step - loss: 0.7375 - accuracy: 0.9500 Epoch 209/500 1/1 [==============================] - 0s 2ms/step - loss: 0.7335 - accuracy: 0.9500 Epoch 210/500 1/1 [==============================] - 0s 2ms/step - loss: 0.7296 - accuracy: 0.9500 Epoch 211/500 1/1 [==============================] - 0s 2ms/step - loss: 0.7257 - accuracy: 0.9500 Epoch 212/500 1/1 [==============================] - 0s 2ms/step - loss: 0.7218 - accuracy: 0.9500 Epoch 213/500 1/1 [==============================] - 0s 997us/step - loss: 0.7180 - accuracy: 0.9500 Epoch 214/500 1/1 [==============================] - 0s 983us/step - loss: 0.7141 - accuracy: 0.9500 Epoch 215/500 1/1 [==============================] - 0s 1ms/step - loss: 0.7103 - accuracy: 0.9500 Epoch 216/500 1/1 [==============================] - 0s 994us/step - loss: 0.7065 - accuracy: 0.9500 Epoch 217/500 1/1 [==============================] - 0s 996us/step - loss: 0.7027 - accuracy: 0.9500 Epoch 218/500 1/1 [==============================] - 0s 998us/step - loss: 0.6989 - accuracy: 0.9500 Epoch 219/500 1/1 [==============================] - 0s 1ms/step - loss: 0.6951 - accuracy: 0.9500 Epoch 220/500 1/1 [==============================] - 0s 999us/step - loss: 0.6913 - accuracy: 0.9500 Epoch 221/500 1/1 [==============================] - 0s 993us/step - loss: 0.6876 - accuracy: 0.9500 Epoch 222/500 1/1 [==============================] - 0s 957us/step - loss: 0.6839 - accuracy: 0.9500 Epoch 223/500 1/1 [==============================] - 0s 994us/step - loss: 0.6801 - accuracy: 0.9500 Epoch 224/500 1/1 [==============================] - 0s 0s/step - loss: 0.6764 - accuracy: 0.9500 Epoch 225/500 1/1 [==============================] - 0s 1000us/step - loss: 0.6727 - accuracy: 0.9500 Epoch 226/500 1/1 [==============================] - 0s 2ms/step - loss: 0.6690 - accuracy: 0.9500 Epoch 227/500 1/1 [==============================] - 0s 988us/step - loss: 0.6654 - accuracy: 0.9500 Epoch 228/500 1/1 [==============================] - 0s 1ms/step - loss: 0.6618 - accuracy: 0.9500 Epoch 229/500 1/1 [==============================] - 0s 0s/step - loss: 0.6581 - accuracy: 0.9500 Epoch 230/500 1/1 [==============================] - 0s 995us/step - loss: 0.6545 - accuracy: 0.9500 Epoch 231/500 1/1 [==============================] - 0s 2ms/step - loss: 0.6509 - accuracy: 0.9500 Epoch 232/500 1/1 [==============================] - 0s 995us/step - loss: 0.6473 - accuracy: 0.9500 Epoch 233/500 1/1 [==============================] - 0s 2ms/step - loss: 0.6437 - accuracy: 0.9500 Epoch 234/500 1/1 [==============================] - 0s 997us/step - loss: 0.6402 - accuracy: 0.9500 Epoch 235/500 1/1 [==============================] - 0s 997us/step - loss: 0.6366 - accuracy: 0.9500 Epoch 236/500 1/1 [==============================] - 0s 995us/step - loss: 0.6331 - accuracy: 0.9500 Epoch 237/500 1/1 [==============================] - 0s 998us/step - loss: 0.6296 - accuracy: 1.0000 Epoch 238/500 1/1 [==============================] - 0s 995us/step - loss: 0.6261 - accuracy: 1.0000 Epoch 239/500 1/1 [==============================] - 0s 996us/step - loss: 0.6226 - accuracy: 1.0000 Epoch 240/500 1/1 [==============================] - 0s 2ms/step - loss: 0.6191 - accuracy: 1.0000 Epoch 241/500 1/1 [==============================] - 0s 2ms/step - loss: 0.6156 - accuracy: 1.0000 Epoch 242/500 1/1 [==============================] - 0s 2ms/step - loss: 0.6122 - accuracy: 1.0000 Epoch 243/500 1/1 [==============================] - 0s 996us/step - loss: 0.6088 - accuracy: 1.0000 Epoch 244/500 1/1 [==============================] - 0s 1ms/step - loss: 0.6054 - accuracy: 1.0000 Epoch 245/500 1/1 [==============================] - 0s 996us/step - loss: 0.6020 - accuracy: 1.0000 Epoch 246/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5986 - accuracy: 1.0000 Epoch 247/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5952 - accuracy: 1.0000 Epoch 248/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5918 - accuracy: 1.0000 Epoch 249/500 1/1 [==============================] - 0s 999us/step - loss: 0.5885 - accuracy: 1.0000 Epoch 250/500 1/1 [==============================] - 0s 1ms/step - loss: 0.5852 - accuracy: 1.0000 Epoch 251/500 1/1 [==============================] - 0s 998us/step - loss: 0.5819 - accuracy: 1.0000 Epoch 252/500 1/1 [==============================] - 0s 997us/step - loss: 0.5785 - accuracy: 1.0000 Epoch 253/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5753 - accuracy: 1.0000 Epoch 254/500 1/1 [==============================] - 0s 997us/step - loss: 0.5720 - accuracy: 1.0000 Epoch 255/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5688 - accuracy: 1.0000 Epoch 256/500 1/1 [==============================] - 0s 997us/step - loss: 0.5655 - accuracy: 1.0000 Epoch 257/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5623 - accuracy: 1.0000 Epoch 258/500 1/1 [==============================] - 0s 998us/step - loss: 0.5591 - accuracy: 1.0000 Epoch 259/500 1/1 [==============================] - 0s 996us/step - loss: 0.5559 - accuracy: 1.0000 Epoch 260/500 1/1 [==============================] - 0s 985us/step - loss: 0.5526 - accuracy: 1.0000 Epoch 261/500 1/1 [==============================] - 0s 997us/step - loss: 0.5495 - accuracy: 1.0000 Epoch 262/500 1/1 [==============================] - 0s 994us/step - loss: 0.5463 - accuracy: 1.0000 Epoch 263/500 1/1 [==============================] - 0s 996us/step - loss: 0.5431 - accuracy: 1.0000 Epoch 264/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5400 - accuracy: 1.0000 Epoch 265/500 1/1 [==============================] - 0s 994us/step - loss: 0.5369 - accuracy: 1.0000 Epoch 266/500 1/1 [==============================] - 0s 995us/step - loss: 0.5337 - accuracy: 1.0000 Epoch 267/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5306 - accuracy: 1.0000 Epoch 268/500 1/1 [==============================] - 0s 997us/step - loss: 0.5275 - accuracy: 1.0000 Epoch 269/500 1/1 [==============================] - 0s 998us/step - loss: 0.5243 - accuracy: 1.0000 Epoch 270/500 1/1 [==============================] - 0s 999us/step - loss: 0.5212 - accuracy: 1.0000 Epoch 271/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5182 - accuracy: 1.0000 Epoch 272/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5151 - accuracy: 1.0000 Epoch 273/500 1/1 [==============================] - 0s 988us/step - loss: 0.5121 - accuracy: 1.0000 Epoch 274/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5090 - accuracy: 1.0000 Epoch 275/500 1/1 [==============================] - 0s 995us/step - loss: 0.5060 - accuracy: 1.0000 Epoch 276/500 1/1 [==============================] - 0s 1ms/step - loss: 0.5030 - accuracy: 1.0000 Epoch 277/500 1/1 [==============================] - 0s 2ms/step - loss: 0.5000 - accuracy: 1.0000 Epoch 278/500 1/1 [==============================] - 0s 996us/step - loss: 0.4971 - accuracy: 1.0000 Epoch 279/500 1/1 [==============================] - 0s 996us/step - loss: 0.4941 - accuracy: 1.0000 Epoch 280/500 1/1 [==============================] - 0s 994us/step - loss: 0.4912 - accuracy: 1.0000 Epoch 281/500 1/1 [==============================] - 0s 999us/step - loss: 0.4883 - accuracy: 1.0000 Epoch 282/500 1/1 [==============================] - 0s 957us/step - loss: 0.4854 - accuracy: 1.0000 Epoch 283/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4825 - accuracy: 1.0000 Epoch 284/500 1/1 [==============================] - 0s 993us/step - loss: 0.4796 - accuracy: 1.0000 Epoch 285/500 1/1 [==============================] - 0s 999us/step - loss: 0.4768 - accuracy: 1.0000 Epoch 286/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4740 - accuracy: 1.0000 Epoch 287/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4711 - accuracy: 1.0000 Epoch 288/500 1/1 [==============================] - 0s 997us/step - loss: 0.4683 - accuracy: 1.0000 Epoch 289/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4655 - accuracy: 1.0000 Epoch 290/500 1/1 [==============================] - 0s 1ms/step - loss: 0.4628 - accuracy: 1.0000 Epoch 291/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4600 - accuracy: 1.0000 Epoch 292/500 1/1 [==============================] - 0s 995us/step - loss: 0.4573 - accuracy: 1.0000 Epoch 293/500 1/1 [==============================] - 0s 996us/step - loss: 0.4545 - accuracy: 1.0000 Epoch 294/500 1/1 [==============================] - 0s 998us/step - loss: 0.4518 - accuracy: 1.0000 Epoch 295/500 1/1 [==============================] - 0s 1ms/step - loss: 0.4491 - accuracy: 1.0000 Epoch 296/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4465 - accuracy: 1.0000 Epoch 297/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4438 - accuracy: 1.0000 Epoch 298/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4411 - accuracy: 1.0000 Epoch 299/500 1/1 [==============================] - 0s 996us/step - loss: 0.4385 - accuracy: 1.0000 Epoch 300/500 1/1 [==============================] - 0s 996us/step - loss: 0.4359 - accuracy: 1.0000 Epoch 301/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4333 - accuracy: 1.0000 Epoch 302/500 1/1 [==============================] - 0s 995us/step - loss: 0.4307 - accuracy: 1.0000 Epoch 303/500 1/1 [==============================] - 0s 997us/step - loss: 0.4281 - accuracy: 1.0000 Epoch 304/500 1/1 [==============================] - 0s 997us/step - loss: 0.4255 - accuracy: 1.0000 Epoch 305/500 1/1 [==============================] - 0s 971us/step - loss: 0.4229 - accuracy: 1.0000 Epoch 306/500 1/1 [==============================] - 0s 996us/step - loss: 0.4203 - accuracy: 1.0000 Epoch 307/500 1/1 [==============================] - 0s 1000us/step - loss: 0.4178 - accuracy: 1.0000 Epoch 308/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4152 - accuracy: 1.0000 Epoch 309/500 1/1 [==============================] - 0s 996us/step - loss: 0.4127 - accuracy: 1.0000 Epoch 310/500 1/1 [==============================] - 0s 995us/step - loss: 0.4101 - accuracy: 1.0000 Epoch 311/500 1/1 [==============================] - 0s 2ms/step - loss: 0.4076 - accuracy: 1.0000 Epoch 312/500 1/1 [==============================] - 0s 1ms/step - loss: 0.4051 - accuracy: 1.0000 Epoch 313/500 1/1 [==============================] - 0s 3ms/step - loss: 0.4026 - accuracy: 1.0000 Epoch 314/500 1/1 [==============================] - 0s 999us/step - loss: 0.4001 - accuracy: 1.0000 Epoch 315/500 1/1 [==============================] - 0s 993us/step - loss: 0.3976 - accuracy: 1.0000 Epoch 316/500 1/1 [==============================] - 0s 995us/step - loss: 0.3952 - accuracy: 1.0000 Epoch 317/500 1/1 [==============================] - 0s 999us/step - loss: 0.3927 - accuracy: 1.0000 Epoch 318/500 1/1 [==============================] - 0s 994us/step - loss: 0.3903 - accuracy: 1.0000 Epoch 319/500 1/1 [==============================] - 0s 998us/step - loss: 0.3879 - accuracy: 1.0000 Epoch 320/500 1/1 [==============================] - 0s 996us/step - loss: 0.3855 - accuracy: 1.0000 Epoch 321/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3832 - accuracy: 1.0000 Epoch 322/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3808 - accuracy: 1.0000 Epoch 323/500 1/1 [==============================] - 0s 997us/step - loss: 0.3785 - accuracy: 1.0000 Epoch 324/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3761 - accuracy: 1.0000 Epoch 325/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3738 - accuracy: 1.0000 Epoch 326/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3715 - accuracy: 1.0000 Epoch 327/500 1/1 [==============================] - 0s 997us/step - loss: 0.3692 - accuracy: 1.0000 Epoch 328/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3669 - accuracy: 1.0000 Epoch 329/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3646 - accuracy: 1.0000 Epoch 330/500 1/1 [==============================] - 0s 996us/step - loss: 0.3623 - accuracy: 1.0000 Epoch 331/500 1/1 [==============================] - 0s 997us/step - loss: 0.3601 - accuracy: 1.0000 Epoch 332/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3578 - accuracy: 1.0000 Epoch 333/500 1/1 [==============================] - 0s 997us/step - loss: 0.3556 - accuracy: 1.0000 Epoch 334/500 1/1 [==============================] - 0s 994us/step - loss: 0.3534 - accuracy: 1.0000 Epoch 335/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3512 - accuracy: 1.0000 Epoch 336/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3490 - accuracy: 1.0000 Epoch 337/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3469 - accuracy: 1.0000 Epoch 338/500 1/1 [==============================] - 0s 981us/step - loss: 0.3447 - accuracy: 1.0000 Epoch 339/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3426 - accuracy: 1.0000 Epoch 340/500 1/1 [==============================] - 0s 1000us/step - loss: 0.3404 - accuracy: 1.0000 Epoch 341/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3383 - accuracy: 1.0000 Epoch 342/500 1/1 [==============================] - 0s 997us/step - loss: 0.3362 - accuracy: 1.0000 Epoch 343/500 1/1 [==============================] - 0s 998us/step - loss: 0.3341 - accuracy: 1.0000 Epoch 344/500 1/1 [==============================] - 0s 996us/step - loss: 0.3320 - accuracy: 1.0000 Epoch 345/500 1/1 [==============================] - 0s 997us/step - loss: 0.3300 - accuracy: 1.0000 Epoch 346/500 1/1 [==============================] - 0s 997us/step - loss: 0.3279 - accuracy: 1.0000 Epoch 347/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3259 - accuracy: 1.0000 Epoch 348/500 1/1 [==============================] - 0s 996us/step - loss: 0.3238 - accuracy: 1.0000 Epoch 349/500 1/1 [==============================] - 0s 976us/step - loss: 0.3218 - accuracy: 1.0000 Epoch 350/500 1/1 [==============================] - 0s 996us/step - loss: 0.3198 - accuracy: 1.0000 Epoch 351/500 1/1 [==============================] - 0s 997us/step - loss: 0.3178 - accuracy: 1.0000 Epoch 352/500 1/1 [==============================] - 0s 988us/step - loss: 0.3158 - accuracy: 1.0000 Epoch 353/500 1/1 [==============================] - 0s 997us/step - loss: 0.3139 - accuracy: 1.0000 Epoch 354/500 1/1 [==============================] - 0s 994us/step - loss: 0.3120 - accuracy: 1.0000 Epoch 355/500 1/1 [==============================] - 0s 992us/step - loss: 0.3100 - accuracy: 1.0000 Epoch 356/500 1/1 [==============================] - 0s 997us/step - loss: 0.3081 - accuracy: 1.0000 Epoch 357/500 1/1 [==============================] - 0s 995us/step - loss: 0.3063 - accuracy: 1.0000 Epoch 358/500 1/1 [==============================] - 0s 2ms/step - loss: 0.3044 - accuracy: 1.0000 Epoch 359/500 1/1 [==============================] - 0s 996us/step - loss: 0.3026 - accuracy: 1.0000 Epoch 360/500 1/1 [==============================] - 0s 996us/step - loss: 0.3007 - accuracy: 1.0000 Epoch 361/500 1/1 [==============================] - 0s 995us/step - loss: 0.2989 - accuracy: 1.0000 Epoch 362/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2971 - accuracy: 1.0000 Epoch 363/500 1/1 [==============================] - 0s 996us/step - loss: 0.2954 - accuracy: 1.0000 Epoch 364/500 1/1 [==============================] - 0s 995us/step - loss: 0.2936 - accuracy: 1.0000 Epoch 365/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2918 - accuracy: 1.0000 Epoch 366/500 1/1 [==============================] - 0s 998us/step - loss: 0.2901 - accuracy: 1.0000 Epoch 367/500 1/1 [==============================] - 0s 997us/step - loss: 0.2884 - accuracy: 1.0000 Epoch 368/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2866 - accuracy: 1.0000 Epoch 369/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2849 - accuracy: 1.0000 Epoch 370/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2832 - accuracy: 1.0000 Epoch 371/500 1/1 [==============================] - 0s 1ms/step - loss: 0.2816 - accuracy: 1.0000 Epoch 372/500 1/1 [==============================] - 0s 996us/step - loss: 0.2799 - accuracy: 1.0000 Epoch 373/500 1/1 [==============================] - 0s 995us/step - loss: 0.2782 - accuracy: 1.0000 Epoch 374/500 1/1 [==============================] - 0s 996us/step - loss: 0.2766 - accuracy: 1.0000 Epoch 375/500 1/1 [==============================] - 0s 999us/step - loss: 0.2749 - accuracy: 1.0000 Epoch 376/500 1/1 [==============================] - 0s 995us/step - loss: 0.2733 - accuracy: 1.0000 Epoch 377/500 1/1 [==============================] - 0s 1ms/step - loss: 0.2717 - accuracy: 1.0000 Epoch 378/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2701 - accuracy: 1.0000 Epoch 379/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2685 - accuracy: 1.0000 Epoch 380/500 1/1 [==============================] - 0s 998us/step - loss: 0.2669 - accuracy: 1.0000 Epoch 381/500 1/1 [==============================] - 0s 996us/step - loss: 0.2654 - accuracy: 1.0000 Epoch 382/500 1/1 [==============================] - 0s 996us/step - loss: 0.2638 - accuracy: 1.0000 Epoch 383/500 1/1 [==============================] - 0s 996us/step - loss: 0.2623 - accuracy: 1.0000 Epoch 384/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2607 - accuracy: 1.0000 Epoch 385/500 1/1 [==============================] - 0s 3ms/step - loss: 0.2592 - accuracy: 1.0000 Epoch 386/500 1/1 [==============================] - 0s 995us/step - loss: 0.2577 - accuracy: 1.0000 Epoch 387/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2562 - accuracy: 1.0000 Epoch 388/500 1/1 [==============================] - 0s 997us/step - loss: 0.2547 - accuracy: 1.0000 Epoch 389/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2532 - accuracy: 1.0000 Epoch 390/500 1/1 [==============================] - 0s 1ms/step - loss: 0.2518 - accuracy: 1.0000 Epoch 391/500 1/1 [==============================] - 0s 951us/step - loss: 0.2503 - accuracy: 1.0000 Epoch 392/500 1/1 [==============================] - 0s 0s/step - loss: 0.2489 - accuracy: 1.0000 Epoch 393/500 1/1 [==============================] - 0s 995us/step - loss: 0.2474 - accuracy: 1.0000 Epoch 394/500 1/1 [==============================] - 0s 997us/step - loss: 0.2460 - accuracy: 1.0000 Epoch 395/500 1/1 [==============================] - 0s 997us/step - loss: 0.2446 - accuracy: 1.0000 Epoch 396/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2432 - accuracy: 1.0000 Epoch 397/500 1/1 [==============================] - 0s 997us/step - loss: 0.2418 - accuracy: 1.0000 Epoch 398/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2404 - accuracy: 1.0000 Epoch 399/500 1/1 [==============================] - 0s 995us/step - loss: 0.2390 - accuracy: 1.0000 Epoch 400/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2377 - accuracy: 1.0000 Epoch 401/500 1/1 [==============================] - 0s 996us/step - loss: 0.2363 - accuracy: 1.0000 Epoch 402/500 1/1 [==============================] - 0s 994us/step - loss: 0.2350 - accuracy: 1.0000 Epoch 403/500 1/1 [==============================] - 0s 996us/step - loss: 0.2336 - accuracy: 1.0000 Epoch 404/500 1/1 [==============================] - 0s 1ms/step - loss: 0.2323 - accuracy: 1.0000 Epoch 405/500 1/1 [==============================] - 0s 995us/step - loss: 0.2310 - accuracy: 1.0000 Epoch 406/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2297 - accuracy: 1.0000 Epoch 407/500 1/1 [==============================] - 0s 1ms/step - loss: 0.2284 - accuracy: 1.0000 Epoch 408/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2271 - accuracy: 1.0000 Epoch 409/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2258 - accuracy: 1.0000 Epoch 410/500 1/1 [==============================] - 0s 995us/step - loss: 0.2245 - accuracy: 1.0000 Epoch 411/500 1/1 [==============================] - 0s 1ms/step - loss: 0.2232 - accuracy: 1.0000 Epoch 412/500 1/1 [==============================] - 0s 997us/step - loss: 0.2220 - accuracy: 1.0000 Epoch 413/500 1/1 [==============================] - 0s 995us/step - loss: 0.2207 - accuracy: 1.0000 Epoch 414/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2195 - accuracy: 1.0000 Epoch 415/500 1/1 [==============================] - 0s 998us/step - loss: 0.2182 - accuracy: 1.0000 Epoch 416/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2170 - accuracy: 1.0000 Epoch 417/500 1/1 [==============================] - 0s 997us/step - loss: 0.2158 - accuracy: 1.0000 Epoch 418/500 1/1 [==============================] - 0s 996us/step - loss: 0.2145 - accuracy: 1.0000 Epoch 419/500 1/1 [==============================] - 0s 1ms/step - loss: 0.2133 - accuracy: 1.0000 Epoch 420/500 1/1 [==============================] - 0s 997us/step - loss: 0.2121 - accuracy: 1.0000 Epoch 421/500 1/1 [==============================] - 0s 992us/step - loss: 0.2109 - accuracy: 1.0000 Epoch 422/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2097 - accuracy: 1.0000 Epoch 423/500 1/1 [==============================] - 0s 1ms/step - loss: 0.2086 - accuracy: 1.0000 Epoch 424/500 1/1 [==============================] - 0s 997us/step - loss: 0.2074 - accuracy: 1.0000 Epoch 425/500 1/1 [==============================] - 0s 994us/step - loss: 0.2062 - accuracy: 1.0000 Epoch 426/500 1/1 [==============================] - 0s 2ms/step - loss: 0.2051 - accuracy: 1.0000 Epoch 427/500 1/1 [==============================] - 0s 997us/step - loss: 0.2040 - accuracy: 1.0000 Epoch 428/500 1/1 [==============================] - 0s 996us/step - loss: 0.2029 - accuracy: 1.0000 Epoch 429/500 1/1 [==============================] - 0s 998us/step - loss: 0.2018 - accuracy: 1.0000 Epoch 430/500 1/1 [==============================] - 0s 995us/step - loss: 0.2007 - accuracy: 1.0000 Epoch 431/500 1/1 [==============================] - 0s 986us/step - loss: 0.1996 - accuracy: 1.0000 Epoch 432/500 1/1 [==============================] - 0s 996us/step - loss: 0.1985 - accuracy: 1.0000 Epoch 433/500 1/1 [==============================] - 0s 997us/step - loss: 0.1974 - accuracy: 1.0000 Epoch 434/500 1/1 [==============================] - 0s 994us/step - loss: 0.1964 - accuracy: 1.0000 Epoch 435/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1953 - accuracy: 1.0000 Epoch 436/500 1/1 [==============================] - 0s 996us/step - loss: 0.1943 - accuracy: 1.0000 Epoch 437/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1932 - accuracy: 1.0000 Epoch 438/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1922 - accuracy: 1.0000 Epoch 439/500 1/1 [==============================] - 0s 994us/step - loss: 0.1911 - accuracy: 1.0000 Epoch 440/500 1/1 [==============================] - 0s 962us/step - loss: 0.1901 - accuracy: 1.0000 Epoch 441/500 1/1 [==============================] - 0s 997us/step - loss: 0.1891 - accuracy: 1.0000 Epoch 442/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1881 - accuracy: 1.0000 Epoch 443/500 1/1 [==============================] - 0s 996us/step - loss: 0.1871 - accuracy: 1.0000 Epoch 444/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1861 - accuracy: 1.0000 Epoch 445/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1851 - accuracy: 1.0000 Epoch 446/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1841 - accuracy: 1.0000 Epoch 447/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1832 - accuracy: 1.0000 Epoch 448/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1822 - accuracy: 1.0000 Epoch 449/500 1/1 [==============================] - 0s 997us/step - loss: 0.1812 - accuracy: 1.0000 Epoch 450/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1803 - accuracy: 1.0000 Epoch 451/500 1/1 [==============================] - 0s 1000us/step - loss: 0.1793 - accuracy: 1.0000 Epoch 452/500 1/1 [==============================] - 0s 1ms/step - loss: 0.1784 - accuracy: 1.0000 Epoch 453/500 1/1 [==============================] - 0s 998us/step - loss: 0.1774 - accuracy: 1.0000 Epoch 454/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1765 - accuracy: 1.0000 Epoch 455/500 1/1 [==============================] - 0s 999us/step - loss: 0.1756 - accuracy: 1.0000 Epoch 456/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1747 - accuracy: 1.0000 Epoch 457/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1738 - accuracy: 1.0000 Epoch 458/500 1/1 [==============================] - 0s 997us/step - loss: 0.1729 - accuracy: 1.0000 Epoch 459/500 1/1 [==============================] - 0s 1ms/step - loss: 0.1720 - accuracy: 1.0000 Epoch 460/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1711 - accuracy: 1.0000 Epoch 461/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1702 - accuracy: 1.0000 Epoch 462/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1694 - accuracy: 1.0000 Epoch 463/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1685 - accuracy: 1.0000 Epoch 464/500 1/1 [==============================] - 0s 1ms/step - loss: 0.1676 - accuracy: 1.0000 Epoch 465/500 1/1 [==============================] - 0s 997us/step - loss: 0.1668 - accuracy: 1.0000 Epoch 466/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1659 - accuracy: 1.0000 Epoch 467/500 1/1 [==============================] - 0s 991us/step - loss: 0.1651 - accuracy: 1.0000 Epoch 468/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1642 - accuracy: 1.0000 Epoch 469/500 1/1 [==============================] - 0s 994us/step - loss: 0.1634 - accuracy: 1.0000 Epoch 470/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1626 - accuracy: 1.0000 Epoch 471/500 1/1 [==============================] - 0s 990us/step - loss: 0.1618 - accuracy: 1.0000 Epoch 472/500 1/1 [==============================] - 0s 961us/step - loss: 0.1609 - accuracy: 1.0000 Epoch 473/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1601 - accuracy: 1.0000 Epoch 474/500 1/1 [==============================] - 0s 996us/step - loss: 0.1593 - accuracy: 1.0000 Epoch 475/500 1/1 [==============================] - 0s 1ms/step - loss: 0.1585 - accuracy: 1.0000 Epoch 476/500 1/1 [==============================] - 0s 996us/step - loss: 0.1577 - accuracy: 1.0000 Epoch 477/500 1/1 [==============================] - 0s 994us/step - loss: 0.1570 - accuracy: 1.0000 Epoch 478/500 1/1 [==============================] - 0s 998us/step - loss: 0.1562 - accuracy: 1.0000 Epoch 479/500 1/1 [==============================] - 0s 995us/step - loss: 0.1554 - accuracy: 1.0000 Epoch 480/500 1/1 [==============================] - 0s 996us/step - loss: 0.1546 - accuracy: 1.0000 Epoch 481/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1539 - accuracy: 1.0000 Epoch 482/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1531 - accuracy: 1.0000 Epoch 483/500 1/1 [==============================] - 0s 997us/step - loss: 0.1523 - accuracy: 1.0000 Epoch 484/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1516 - accuracy: 1.0000 Epoch 485/500 1/1 [==============================] - 0s 997us/step - loss: 0.1508 - accuracy: 1.0000 Epoch 486/500 1/1 [==============================] - 0s 997us/step - loss: 0.1501 - accuracy: 1.0000 Epoch 487/500 1/1 [==============================] - 0s 952us/step - loss: 0.1494 - accuracy: 1.0000 Epoch 488/500 1/1 [==============================] - 0s 999us/step - loss: 0.1486 - accuracy: 1.0000 Epoch 489/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1479 - accuracy: 1.0000 Epoch 490/500 1/1 [==============================] - 0s 995us/step - loss: 0.1472 - accuracy: 1.0000 Epoch 491/500 1/1 [==============================] - 0s 2ms/step - loss: 0.1465 - accuracy: 1.0000 Epoch 492/500 1/1 [==============================] - 0s 999us/step - loss: 0.1458 - accuracy: 1.0000 Epoch 493/500 1/1 [==============================] - 0s 997us/step - loss: 0.1451 - accuracy: 1.0000 Epoch 494/500 1/1 [==============================] - 0s 996us/step - loss: 0.1444 - accuracy: 1.0000 Epoch 495/500 1/1 [==============================] - 0s 1ms/step - loss: 0.1437 - accuracy: 1.0000 Epoch 496/500 1/1 [==============================] - 0s 997us/step - loss: 0.1430 - accuracy: 1.0000 Epoch 497/500 1/1 [==============================] - 0s 997us/step - loss: 0.1423 - accuracy: 1.0000 Epoch 498/500 1/1 [==============================] - 0s 997us/step - loss: 0.1416 - accuracy: 1.0000 Epoch 499/500 1/1 [==============================] - 0s 998us/step - loss: 0.1410 - accuracy: 1.0000 Epoch 500/500 1/1 [==============================] - 0s 995us/step - loss: 0.1403 - accuracy: 1.0000
lr = 1e-5
Lambda = 1e-7
train_and_test_loop(20, lr, Lambda)
Epoch 1/20 42/42 [==============================] - 1s 20ms/step - loss: 2.3833 - accuracy: 0.0986 Epoch 2/20 42/42 [==============================] - 1s 19ms/step - loss: 2.3737 - accuracy: 0.0988 Epoch 3/20 42/42 [==============================] - 1s 20ms/step - loss: 2.3650 - accuracy: 0.0985 Epoch 4/20 42/42 [==============================] - 1s 18ms/step - loss: 2.3574 - accuracy: 0.0985 Epoch 5/20 42/42 [==============================] - 1s 18ms/step - loss: 2.3508 - accuracy: 0.0983 Epoch 6/20 42/42 [==============================] - 1s 19ms/step - loss: 2.3449 - accuracy: 0.0991 Epoch 7/20 42/42 [==============================] - 1s 18ms/step - loss: 2.3398 - accuracy: 0.1013 Epoch 8/20 42/42 [==============================] - 1s 19ms/step - loss: 2.3353 - accuracy: 0.1029 Epoch 9/20 42/42 [==============================] - 1s 19ms/step - loss: 2.3314 - accuracy: 0.1048 Epoch 10/20 42/42 [==============================] - 1s 19ms/step - loss: 2.3279 - accuracy: 0.1068 Epoch 11/20 42/42 [==============================] - 1s 20ms/step - loss: 2.3248 - accuracy: 0.1088 Epoch 12/20 42/42 [==============================] - 1s 20ms/step - loss: 2.3221 - accuracy: 0.1108 Epoch 13/20 42/42 [==============================] - 1s 18ms/step - loss: 2.3197 - accuracy: 0.1132 Epoch 14/20 42/42 [==============================] - 1s 21ms/step - loss: 2.3176 - accuracy: 0.1144 Epoch 15/20 42/42 [==============================] - 1s 18ms/step - loss: 2.3157 - accuracy: 0.1156 Epoch 16/20 42/42 [==============================] - 1s 19ms/step - loss: 2.3141 - accuracy: 0.1163 Epoch 17/20 42/42 [==============================] - 1s 18ms/step - loss: 2.3126 - accuracy: 0.1172 Epoch 18/20 42/42 [==============================] - 1s 18ms/step - loss: 2.3113 - accuracy: 0.1179 Epoch 19/20 42/42 [==============================] - 1s 18ms/step - loss: 2.3102 - accuracy: 0.1184 Epoch 20/20 42/42 [==============================] - 1s 19ms/step - loss: 2.3092 - accuracy: 0.1188
lr = 1e8
Lambda = 1e-7
train_and_test_loop(20, lr, Lambda)
Epoch 1/20 42/42 [==============================] - 1s 19ms/step - loss: nan - accuracy: 0.0995 Epoch 2/20 42/42 [==============================] - 1s 21ms/step - loss: nan - accuracy: 0.0997 Epoch 3/20 42/42 [==============================] - 1s 21ms/step - loss: nan - accuracy: 0.0997 Epoch 4/20 42/42 [==============================] - 1s 18ms/step - loss: nan - accuracy: 0.0997 Epoch 5/20 42/42 [==============================] - 1s 18ms/step - loss: nan - accuracy: 0.0997 Epoch 6/20 42/42 [==============================] - 1s 17ms/step - loss: nan - accuracy: 0.0997 Epoch 7/20 42/42 [==============================] - 1s 21ms/step - loss: nan - accuracy: 0.0997 Epoch 8/20 42/42 [==============================] - 1s 17ms/step - loss: nan - accuracy: 0.0997 Epoch 9/20 42/42 [==============================] - 1s 16ms/step - loss: nan - accuracy: 0.0997 Epoch 10/20 42/42 [==============================] - 1s 16ms/step - loss: nan - accuracy: 0.0997 Epoch 11/20 42/42 [==============================] - 1s 16ms/step - loss: nan - accuracy: 0.0997 Epoch 12/20 42/42 [==============================] - 1s 16ms/step - loss: nan - accuracy: 0.0997 Epoch 13/20 42/42 [==============================] - 1s 16ms/step - loss: nan - accuracy: 0.0997 Epoch 14/20 42/42 [==============================] - 1s 16ms/step - loss: nan - accuracy: 0.0997 Epoch 15/20 42/42 [==============================] - 1s 16ms/step - loss: nan - accuracy: 0.0997 Epoch 16/20 42/42 [==============================] - 1s 19ms/step - loss: nan - accuracy: 0.0997 Epoch 17/20 42/42 [==============================] - 1s 18ms/step - loss: nan - accuracy: 0.0997 Epoch 18/20 42/42 [==============================] - 1s 16ms/step - loss: nan - accuracy: 0.0997 Epoch 19/20 42/42 [==============================] - 1s 16ms/step - loss: nan - accuracy: 0.0997 Epoch 20/20 42/42 [==============================] - 1s 17ms/step - loss: nan - accuracy: 0.0997
lr = 1e-1
Lambda = 1e-7
train_and_test_loop(20, lr, Lambda)
Epoch 1/20 42/42 [==============================] - 1s 19ms/step - loss: 2.2961 - accuracy: 0.1393 Epoch 2/20 42/42 [==============================] - 1s 18ms/step - loss: 2.0985 - accuracy: 0.2742 Epoch 3/20 42/42 [==============================] - 1s 18ms/step - loss: 2.2070 - accuracy: 0.2713 Epoch 4/20 42/42 [==============================] - 1s 19ms/step - loss: 2.0607 - accuracy: 0.2755 Epoch 5/20 42/42 [==============================] - 1s 18ms/step - loss: 2.1432 - accuracy: 0.2175 Epoch 6/20 42/42 [==============================] - 1s 19ms/step - loss: 2.1518 - accuracy: 0.1935 Epoch 7/20 42/42 [==============================] - 1s 19ms/step - loss: 2.0752 - accuracy: 0.2351 Epoch 8/20 42/42 [==============================] - 1s 18ms/step - loss: 2.2189 - accuracy: 0.1578 Epoch 9/20 42/42 [==============================] - 1s 19ms/step - loss: 2.2661 - accuracy: 0.1427 Epoch 10/20 42/42 [==============================] - 1s 19ms/step - loss: 2.2083 - accuracy: 0.1676 Epoch 11/20 42/42 [==============================] - 1s 22ms/step - loss: 2.2284 - accuracy: 0.1410 Epoch 12/20 42/42 [==============================] - 1s 22ms/step - loss: 2.1633 - accuracy: 0.1738 Epoch 13/20 42/42 [==============================] - 1s 20ms/step - loss: 2.1775 - accuracy: 0.1698 Epoch 14/20 42/42 [==============================] - 1s 20ms/step - loss: 2.1332 - accuracy: 0.1899 Epoch 15/20 42/42 [==============================] - 1s 18ms/step - loss: 2.0041 - accuracy: 0.2365 Epoch 16/20 42/42 [==============================] - 1s 18ms/step - loss: 1.9487 - accuracy: 0.2755 Epoch 17/20 42/42 [==============================] - 1s 22ms/step - loss: 1.9343 - accuracy: 0.2919 Epoch 18/20 42/42 [==============================] - 1s 20ms/step - loss: 1.9099 - accuracy: 0.2970 Epoch 19/20 42/42 [==============================] - 1s 20ms/step - loss: 1.8860 - accuracy: 0.3102 Epoch 20/20 42/42 [==============================] - 1s 19ms/step - loss: 1.8897 - accuracy: 0.3084
for k in range(1,10):
lr = np.random.uniform(0.01, 0.1)
Lambda = 1.3190300180182324e-05
# math.pow(10, np.random.uniform(-7,-2))
best_acc = train_and_test_loop1(100, lr, Lambda, False)
print("Try {0}/{1}: Best_val_acc: {2}, lr: {3}, Lambda: {4}\n".format(k, 9, best_acc, lr, Lambda))
Try 1/9: Best_val_acc: [0.42673882842063904, 0.8680475950241089], lr: 0.09508597207038409, Lambda: 1.3190300180182324e-05 Try 2/9: Best_val_acc: [0.3390926718711853, 0.899404764175415], lr: 0.02386272523187538, Lambda: 1.3190300180182324e-05 Try 3/9: Best_val_acc: [0.23015905916690826, 0.9267619252204895], lr: 0.06951722705476646, Lambda: 1.3190300180182324e-05 Try 4/9: Best_val_acc: [0.3857276141643524, 0.8785476088523865], lr: 0.07461987672442802, Lambda: 1.3190300180182324e-05 Try 5/9: Best_val_acc: [0.24405735731124878, 0.9240238070487976], lr: 0.08657951857639976, Lambda: 1.3190300180182324e-05 Try 6/9: Best_val_acc: [0.2613851726055145, 0.9215952157974243], lr: 0.03401920453424891, Lambda: 1.3190300180182324e-05 Try 7/9: Best_val_acc: [0.6123517751693726, 0.8116190433502197], lr: 0.09763818514424291, Lambda: 1.3190300180182324e-05 Try 8/9: Best_val_acc: [0.3451426327228546, 0.899142861366272], lr: 0.018966539181242192, Lambda: 1.3190300180182324e-05 Try 9/9: Best_val_acc: [0.22561976313591003, 0.9314285516738892], lr: 0.0589421087034557, Lambda: 1.3190300180182324e-05
Lambda = 1.5e-04
## hyperparameters
iterations = 150
learning_rate = 0.0589421087034557
hidden_nodes = 339
output_nodes = 10
model = Sequential()
model.add(Dense(hidden_nodes, input_shape=(1024,), activation='relu'))
model.add(Dense(hidden_nodes, activation='relu'))
model.add(Dense(output_nodes, activation='softmax', kernel_regularizer=regularizers.l2(Lambda)))
sgd = optimizers.SGD(lr=learning_rate, decay=1e-6, momentum=0.9)
# Compile model
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
# Fit the model
hist = model.fit(X_train, y_train, epochs=iterations, batch_size=1000,
verbose=2, validation_data=(X_val, y_val))
Epoch 1/150 60/60 - 1s - loss: 2.2702 - accuracy: 0.1725 - val_loss: 2.1464 - val_accuracy: 0.3373 Epoch 2/150 60/60 - 1s - loss: 1.8702 - accuracy: 0.3972 - val_loss: 1.6748 - val_accuracy: 0.4227 Epoch 3/150 60/60 - 1s - loss: 1.5044 - accuracy: 0.5080 - val_loss: 1.3885 - val_accuracy: 0.5410 Epoch 4/150 60/60 - 1s - loss: 1.2815 - accuracy: 0.5899 - val_loss: 1.2398 - val_accuracy: 0.6029 Epoch 5/150 60/60 - 1s - loss: 1.1706 - accuracy: 0.6309 - val_loss: 1.1259 - val_accuracy: 0.6501 Epoch 6/150 60/60 - 1s - loss: 1.0593 - accuracy: 0.6730 - val_loss: 1.0446 - val_accuracy: 0.6688 Epoch 7/150 60/60 - 1s - loss: 0.9721 - accuracy: 0.7018 - val_loss: 0.9005 - val_accuracy: 0.7358 Epoch 8/150 60/60 - 1s - loss: 0.9259 - accuracy: 0.7182 - val_loss: 0.9432 - val_accuracy: 0.7065 Epoch 9/150 60/60 - 1s - loss: 0.8831 - accuracy: 0.7311 - val_loss: 0.8567 - val_accuracy: 0.7397 Epoch 10/150 60/60 - 1s - loss: 0.8282 - accuracy: 0.7496 - val_loss: 0.8083 - val_accuracy: 0.7544 Epoch 11/150 60/60 - 1s - loss: 0.7932 - accuracy: 0.7616 - val_loss: 0.8015 - val_accuracy: 0.7525 Epoch 12/150 60/60 - 1s - loss: 0.7724 - accuracy: 0.7689 - val_loss: 0.7459 - val_accuracy: 0.7760 Epoch 13/150 60/60 - 1s - loss: 0.7715 - accuracy: 0.7682 - val_loss: 0.7811 - val_accuracy: 0.7633 Epoch 14/150 60/60 - 1s - loss: 0.7241 - accuracy: 0.7839 - val_loss: 0.7214 - val_accuracy: 0.7861 Epoch 15/150 60/60 - 1s - loss: 0.7089 - accuracy: 0.7896 - val_loss: 0.7347 - val_accuracy: 0.7756 Epoch 16/150 60/60 - 1s - loss: 0.6933 - accuracy: 0.7941 - val_loss: 0.6455 - val_accuracy: 0.8135 Epoch 17/150 60/60 - 1s - loss: 0.6638 - accuracy: 0.8055 - val_loss: 0.6311 - val_accuracy: 0.8167 Epoch 18/150 60/60 - 1s - loss: 0.6609 - accuracy: 0.8050 - val_loss: 0.6555 - val_accuracy: 0.8049 Epoch 19/150 60/60 - 1s - loss: 0.6455 - accuracy: 0.8089 - val_loss: 0.6092 - val_accuracy: 0.8223 Epoch 20/150 60/60 - 1s - loss: 0.6109 - accuracy: 0.8226 - val_loss: 0.6018 - val_accuracy: 0.8246 Epoch 21/150 60/60 - 1s - loss: 0.5975 - accuracy: 0.8271 - val_loss: 0.5758 - val_accuracy: 0.8336 Epoch 22/150 60/60 - 1s - loss: 0.5929 - accuracy: 0.8262 - val_loss: 0.5641 - val_accuracy: 0.8363 Epoch 23/150 60/60 - 1s - loss: 0.5773 - accuracy: 0.8312 - val_loss: 0.5632 - val_accuracy: 0.8367 Epoch 24/150 60/60 - 1s - loss: 0.5860 - accuracy: 0.8278 - val_loss: 0.5741 - val_accuracy: 0.8321 Epoch 25/150 60/60 - 1s - loss: 0.5594 - accuracy: 0.8387 - val_loss: 0.5503 - val_accuracy: 0.8381 Epoch 26/150 60/60 - 1s - loss: 0.5694 - accuracy: 0.8330 - val_loss: 0.5532 - val_accuracy: 0.8353 Epoch 27/150 60/60 - 1s - loss: 0.5417 - accuracy: 0.8416 - val_loss: 0.5262 - val_accuracy: 0.8455 Epoch 28/150 60/60 - 1s - loss: 0.5516 - accuracy: 0.8385 - val_loss: 0.5105 - val_accuracy: 0.8538 Epoch 29/150 60/60 - 1s - loss: 0.5296 - accuracy: 0.8454 - val_loss: 0.5359 - val_accuracy: 0.8429 Epoch 30/150 60/60 - 1s - loss: 0.5080 - accuracy: 0.8521 - val_loss: 0.4929 - val_accuracy: 0.8591 Epoch 31/150 60/60 - 1s - loss: 0.5087 - accuracy: 0.8528 - val_loss: 0.5083 - val_accuracy: 0.8500 Epoch 32/150 60/60 - 1s - loss: 0.4936 - accuracy: 0.8576 - val_loss: 0.4721 - val_accuracy: 0.8633 Epoch 33/150 60/60 - 1s - loss: 0.4879 - accuracy: 0.8588 - val_loss: 0.4718 - val_accuracy: 0.8634 Epoch 34/150 60/60 - 1s - loss: 0.4913 - accuracy: 0.8555 - val_loss: 0.4657 - val_accuracy: 0.8650 Epoch 35/150 60/60 - 1s - loss: 0.4852 - accuracy: 0.8587 - val_loss: 0.4660 - val_accuracy: 0.8663 Epoch 36/150 60/60 - 1s - loss: 0.4765 - accuracy: 0.8626 - val_loss: 0.4505 - val_accuracy: 0.8722 Epoch 37/150 60/60 - 1s - loss: 0.4763 - accuracy: 0.8612 - val_loss: 0.4761 - val_accuracy: 0.8601 Epoch 38/150 60/60 - 1s - loss: 0.4699 - accuracy: 0.8644 - val_loss: 0.4543 - val_accuracy: 0.8679 Epoch 39/150 60/60 - 1s - loss: 0.4469 - accuracy: 0.8708 - val_loss: 0.4450 - val_accuracy: 0.8718 Epoch 40/150 60/60 - 1s - loss: 0.4466 - accuracy: 0.8709 - val_loss: 0.4269 - val_accuracy: 0.8775 Epoch 41/150 60/60 - 1s - loss: 0.4462 - accuracy: 0.8706 - val_loss: 0.4458 - val_accuracy: 0.8686 Epoch 42/150 60/60 - 1s - loss: 0.4346 - accuracy: 0.8751 - val_loss: 0.4267 - val_accuracy: 0.8787 Epoch 43/150 60/60 - 1s - loss: 0.4432 - accuracy: 0.8719 - val_loss: 0.4127 - val_accuracy: 0.8809 Epoch 44/150 60/60 - 1s - loss: 0.4338 - accuracy: 0.8751 - val_loss: 0.4106 - val_accuracy: 0.8830 Epoch 45/150 60/60 - 1s - loss: 0.4216 - accuracy: 0.8777 - val_loss: 0.4041 - val_accuracy: 0.8825 Epoch 46/150 60/60 - 1s - loss: 0.4247 - accuracy: 0.8775 - val_loss: 0.4126 - val_accuracy: 0.8795 Epoch 47/150 60/60 - 1s - loss: 0.4161 - accuracy: 0.8794 - val_loss: 0.4343 - val_accuracy: 0.8709 Epoch 48/150 60/60 - 1s - loss: 0.4160 - accuracy: 0.8793 - val_loss: 0.4106 - val_accuracy: 0.8812 Epoch 49/150 60/60 - 1s - loss: 0.4161 - accuracy: 0.8800 - val_loss: 0.3707 - val_accuracy: 0.8962 Epoch 50/150 60/60 - 1s - loss: 0.3956 - accuracy: 0.8857 - val_loss: 0.3774 - val_accuracy: 0.8918 Epoch 51/150 60/60 - 1s - loss: 0.3909 - accuracy: 0.8889 - val_loss: 0.3750 - val_accuracy: 0.8940 Epoch 52/150 60/60 - 1s - loss: 0.3919 - accuracy: 0.8866 - val_loss: 0.3658 - val_accuracy: 0.8947 Epoch 53/150 60/60 - 1s - loss: 0.3887 - accuracy: 0.8881 - val_loss: 0.3704 - val_accuracy: 0.8936 Epoch 54/150 60/60 - 1s - loss: 0.3876 - accuracy: 0.8875 - val_loss: 0.3714 - val_accuracy: 0.8923 Epoch 55/150 60/60 - 1s - loss: 0.3967 - accuracy: 0.8843 - val_loss: 0.3891 - val_accuracy: 0.8870 Epoch 56/150 60/60 - 1s - loss: 0.3847 - accuracy: 0.8880 - val_loss: 0.3674 - val_accuracy: 0.8947 Epoch 57/150 60/60 - 1s - loss: 0.3772 - accuracy: 0.8911 - val_loss: 0.3745 - val_accuracy: 0.8915 Epoch 58/150 60/60 - 1s - loss: 0.3637 - accuracy: 0.8959 - val_loss: 0.3700 - val_accuracy: 0.8930 Epoch 59/150 60/60 - 1s - loss: 0.3781 - accuracy: 0.8898 - val_loss: 0.3546 - val_accuracy: 0.8996 Epoch 60/150 60/60 - 1s - loss: 0.3701 - accuracy: 0.8917 - val_loss: 0.3778 - val_accuracy: 0.8902 Epoch 61/150 60/60 - 1s - loss: 0.3651 - accuracy: 0.8940 - val_loss: 0.3503 - val_accuracy: 0.9010 Epoch 62/150 60/60 - 1s - loss: 0.3504 - accuracy: 0.8996 - val_loss: 0.3570 - val_accuracy: 0.8954 Epoch 63/150 60/60 - 1s - loss: 0.3491 - accuracy: 0.8985 - val_loss: 0.3353 - val_accuracy: 0.9038 Epoch 64/150 60/60 - 1s - loss: 0.3485 - accuracy: 0.9003 - val_loss: 0.3401 - val_accuracy: 0.9013 Epoch 65/150 60/60 - 1s - loss: 0.3387 - accuracy: 0.9033 - val_loss: 0.3360 - val_accuracy: 0.9042 Epoch 66/150 60/60 - 1s - loss: 0.3392 - accuracy: 0.9031 - val_loss: 0.3170 - val_accuracy: 0.9114 Epoch 67/150 60/60 - 1s - loss: 0.3279 - accuracy: 0.9078 - val_loss: 0.3103 - val_accuracy: 0.9154 Epoch 68/150 60/60 - 1s - loss: 0.3363 - accuracy: 0.9022 - val_loss: 0.3334 - val_accuracy: 0.9039 Epoch 69/150 60/60 - 1s - loss: 0.3396 - accuracy: 0.9018 - val_loss: 0.3306 - val_accuracy: 0.9049 Epoch 70/150 60/60 - 1s - loss: 0.3310 - accuracy: 0.9046 - val_loss: 0.3376 - val_accuracy: 0.9012 Epoch 71/150 60/60 - 1s - loss: 0.3281 - accuracy: 0.9059 - val_loss: 0.3110 - val_accuracy: 0.9117 Epoch 72/150 60/60 - 1s - loss: 0.3268 - accuracy: 0.9056 - val_loss: 0.3176 - val_accuracy: 0.9104 Epoch 73/150 60/60 - 1s - loss: 0.3182 - accuracy: 0.9087 - val_loss: 0.2973 - val_accuracy: 0.9180 Epoch 74/150 60/60 - 1s - loss: 0.3105 - accuracy: 0.9120 - val_loss: 0.2979 - val_accuracy: 0.9163 Epoch 75/150 60/60 - 1s - loss: 0.3082 - accuracy: 0.9120 - val_loss: 0.2936 - val_accuracy: 0.9164 Epoch 76/150 60/60 - 1s - loss: 0.3103 - accuracy: 0.9121 - val_loss: 0.2993 - val_accuracy: 0.9149 Epoch 77/150 60/60 - 1s - loss: 0.3082 - accuracy: 0.9106 - val_loss: 0.2886 - val_accuracy: 0.9202 Epoch 78/150 60/60 - 1s - loss: 0.2964 - accuracy: 0.9171 - val_loss: 0.2957 - val_accuracy: 0.9168 Epoch 79/150 60/60 - 1s - loss: 0.3119 - accuracy: 0.9103 - val_loss: 0.2956 - val_accuracy: 0.9169 Epoch 80/150 60/60 - 1s - loss: 0.3108 - accuracy: 0.9105 - val_loss: 0.3020 - val_accuracy: 0.9141 Epoch 81/150 60/60 - 1s - loss: 0.2915 - accuracy: 0.9169 - val_loss: 0.2788 - val_accuracy: 0.9231 Epoch 82/150 60/60 - 1s - loss: 0.2974 - accuracy: 0.9152 - val_loss: 0.2726 - val_accuracy: 0.9235 Epoch 83/150 60/60 - 1s - loss: 0.2898 - accuracy: 0.9170 - val_loss: 0.3210 - val_accuracy: 0.9058 Epoch 84/150 60/60 - 1s - loss: 0.2944 - accuracy: 0.9151 - val_loss: 0.2730 - val_accuracy: 0.9263 Epoch 85/150 60/60 - 1s - loss: 0.2788 - accuracy: 0.9220 - val_loss: 0.2717 - val_accuracy: 0.9242 Epoch 86/150 60/60 - 1s - loss: 0.2844 - accuracy: 0.9195 - val_loss: 0.2742 - val_accuracy: 0.9226 Epoch 87/150 60/60 - 1s - loss: 0.2795 - accuracy: 0.9217 - val_loss: 0.2655 - val_accuracy: 0.9269 Epoch 88/150 60/60 - 1s - loss: 0.2799 - accuracy: 0.9210 - val_loss: 0.2806 - val_accuracy: 0.9201 Epoch 89/150 60/60 - 1s - loss: 0.2814 - accuracy: 0.9201 - val_loss: 0.2681 - val_accuracy: 0.9255 Epoch 90/150 60/60 - 1s - loss: 0.2766 - accuracy: 0.9230 - val_loss: 0.2638 - val_accuracy: 0.9269 Epoch 91/150 60/60 - 1s - loss: 0.2633 - accuracy: 0.9267 - val_loss: 0.2561 - val_accuracy: 0.9294 Epoch 92/150 60/60 - 1s - loss: 0.2651 - accuracy: 0.9262 - val_loss: 0.2664 - val_accuracy: 0.9246 Epoch 93/150 60/60 - 1s - loss: 0.2641 - accuracy: 0.9252 - val_loss: 0.2375 - val_accuracy: 0.9367 Epoch 94/150 60/60 - 1s - loss: 0.2696 - accuracy: 0.9251 - val_loss: 0.2553 - val_accuracy: 0.9300 Epoch 95/150 60/60 - 1s - loss: 0.2592 - accuracy: 0.9281 - val_loss: 0.2447 - val_accuracy: 0.9336 Epoch 96/150 60/60 - 1s - loss: 0.2644 - accuracy: 0.9249 - val_loss: 0.2343 - val_accuracy: 0.9378 Epoch 97/150 60/60 - 1s - loss: 0.2634 - accuracy: 0.9260 - val_loss: 0.2362 - val_accuracy: 0.9359 Epoch 98/150 60/60 - 1s - loss: 0.2522 - accuracy: 0.9300 - val_loss: 0.2336 - val_accuracy: 0.9369 Epoch 99/150 60/60 - 1s - loss: 0.2573 - accuracy: 0.9276 - val_loss: 0.2474 - val_accuracy: 0.9315 Epoch 100/150 60/60 - 1s - loss: 0.2569 - accuracy: 0.9276 - val_loss: 0.2420 - val_accuracy: 0.9342 Epoch 101/150 60/60 - 1s - loss: 0.2586 - accuracy: 0.9269 - val_loss: 0.2575 - val_accuracy: 0.9263 Epoch 102/150 60/60 - 1s - loss: 0.2511 - accuracy: 0.9299 - val_loss: 0.2534 - val_accuracy: 0.9295 Epoch 103/150 60/60 - 1s - loss: 0.2545 - accuracy: 0.9270 - val_loss: 0.2385 - val_accuracy: 0.9349 Epoch 104/150 60/60 - 1s - loss: 0.2422 - accuracy: 0.9336 - val_loss: 0.2343 - val_accuracy: 0.9365 Epoch 105/150 60/60 - 1s - loss: 0.2419 - accuracy: 0.9330 - val_loss: 0.2231 - val_accuracy: 0.9420 Epoch 106/150 60/60 - 1s - loss: 0.2544 - accuracy: 0.9287 - val_loss: 0.2321 - val_accuracy: 0.9357 Epoch 107/150 60/60 - 1s - loss: 0.2419 - accuracy: 0.9331 - val_loss: 0.2156 - val_accuracy: 0.9431 Epoch 108/150 60/60 - 1s - loss: 0.2305 - accuracy: 0.9369 - val_loss: 0.2157 - val_accuracy: 0.9430 Epoch 109/150 60/60 - 1s - loss: 0.2341 - accuracy: 0.9352 - val_loss: 0.2310 - val_accuracy: 0.9375 Epoch 110/150 60/60 - 1s - loss: 0.2338 - accuracy: 0.9344 - val_loss: 0.2165 - val_accuracy: 0.9421 Epoch 111/150 60/60 - 1s - loss: 0.2352 - accuracy: 0.9348 - val_loss: 0.2272 - val_accuracy: 0.9394 Epoch 112/150 60/60 - 1s - loss: 0.2345 - accuracy: 0.9339 - val_loss: 0.2129 - val_accuracy: 0.9423 Epoch 113/150 60/60 - 1s - loss: 0.2274 - accuracy: 0.9375 - val_loss: 0.2350 - val_accuracy: 0.9352 Epoch 114/150 60/60 - 1s - loss: 0.2315 - accuracy: 0.9356 - val_loss: 0.2278 - val_accuracy: 0.9366 Epoch 115/150 60/60 - 1s - loss: 0.2179 - accuracy: 0.9407 - val_loss: 0.2086 - val_accuracy: 0.9462 Epoch 116/150 60/60 - 1s - loss: 0.2195 - accuracy: 0.9403 - val_loss: 0.2089 - val_accuracy: 0.9458 Epoch 117/150 60/60 - 1s - loss: 0.2203 - accuracy: 0.9389 - val_loss: 0.2074 - val_accuracy: 0.9459 Epoch 118/150 60/60 - 1s - loss: 0.2153 - accuracy: 0.9412 - val_loss: 0.1896 - val_accuracy: 0.9520 Epoch 119/150 60/60 - 1s - loss: 0.2150 - accuracy: 0.9418 - val_loss: 0.1937 - val_accuracy: 0.9508 Epoch 120/150 60/60 - 1s - loss: 0.2110 - accuracy: 0.9435 - val_loss: 0.2062 - val_accuracy: 0.9448 Epoch 121/150 60/60 - 1s - loss: 0.2083 - accuracy: 0.9440 - val_loss: 0.2003 - val_accuracy: 0.9480 Epoch 122/150 60/60 - 1s - loss: 0.2187 - accuracy: 0.9397 - val_loss: 0.2204 - val_accuracy: 0.9394 Epoch 123/150 60/60 - 1s - loss: 0.2188 - accuracy: 0.9396 - val_loss: 0.2010 - val_accuracy: 0.9474 Epoch 124/150 60/60 - 1s - loss: 0.2017 - accuracy: 0.9465 - val_loss: 0.2028 - val_accuracy: 0.9450 Epoch 125/150 60/60 - 1s - loss: 0.2064 - accuracy: 0.9442 - val_loss: 0.1995 - val_accuracy: 0.9470 Epoch 126/150 60/60 - 1s - loss: 0.2071 - accuracy: 0.9441 - val_loss: 0.2013 - val_accuracy: 0.9455 Epoch 127/150 60/60 - 1s - loss: 0.1994 - accuracy: 0.9470 - val_loss: 0.2036 - val_accuracy: 0.9453 Epoch 128/150 60/60 - 1s - loss: 0.1984 - accuracy: 0.9483 - val_loss: 0.1909 - val_accuracy: 0.9504 Epoch 129/150 60/60 - 1s - loss: 0.1954 - accuracy: 0.9484 - val_loss: 0.1876 - val_accuracy: 0.9510 Epoch 130/150 60/60 - 1s - loss: 0.2040 - accuracy: 0.9448 - val_loss: 0.1940 - val_accuracy: 0.9486 Epoch 131/150 60/60 - 1s - loss: 0.1952 - accuracy: 0.9490 - val_loss: 0.1994 - val_accuracy: 0.9466 Epoch 132/150 60/60 - 1s - loss: 0.1965 - accuracy: 0.9476 - val_loss: 0.2014 - val_accuracy: 0.9459 Epoch 133/150 60/60 - 1s - loss: 0.2116 - accuracy: 0.9410 - val_loss: 0.1818 - val_accuracy: 0.9543 Epoch 134/150 60/60 - 1s - loss: 0.1882 - accuracy: 0.9501 - val_loss: 0.1818 - val_accuracy: 0.9542 Epoch 135/150 60/60 - 1s - loss: 0.1869 - accuracy: 0.9517 - val_loss: 0.1778 - val_accuracy: 0.9560 Epoch 136/150 60/60 - 1s - loss: 0.1798 - accuracy: 0.9539 - val_loss: 0.1634 - val_accuracy: 0.9608 Epoch 137/150 60/60 - 1s - loss: 0.1848 - accuracy: 0.9519 - val_loss: 0.1779 - val_accuracy: 0.9549 Epoch 138/150 60/60 - 1s - loss: 0.1803 - accuracy: 0.9537 - val_loss: 0.1837 - val_accuracy: 0.9516 Epoch 139/150 60/60 - 1s - loss: 0.1787 - accuracy: 0.9540 - val_loss: 0.1666 - val_accuracy: 0.9587 Epoch 140/150 60/60 - 1s - loss: 0.1809 - accuracy: 0.9530 - val_loss: 0.1862 - val_accuracy: 0.9502 Epoch 141/150 60/60 - 1s - loss: 0.1672 - accuracy: 0.9587 - val_loss: 0.1671 - val_accuracy: 0.9588 Epoch 142/150 60/60 - 1s - loss: 0.1805 - accuracy: 0.9526 - val_loss: 0.1679 - val_accuracy: 0.9579 Epoch 143/150 60/60 - 1s - loss: 0.1733 - accuracy: 0.9557 - val_loss: 0.1884 - val_accuracy: 0.9501 Epoch 144/150 60/60 - 1s - loss: 0.1719 - accuracy: 0.9558 - val_loss: 0.1550 - val_accuracy: 0.9639 Epoch 145/150 60/60 - 1s - loss: 0.1703 - accuracy: 0.9564 - val_loss: 0.1706 - val_accuracy: 0.9566 Epoch 146/150 60/60 - 1s - loss: 0.1780 - accuracy: 0.9530 - val_loss: 0.1749 - val_accuracy: 0.9553 Epoch 147/150 60/60 - 1s - loss: 0.1762 - accuracy: 0.9540 - val_loss: 0.1728 - val_accuracy: 0.9561 Epoch 148/150 60/60 - 1s - loss: 0.1707 - accuracy: 0.9565 - val_loss: 0.1607 - val_accuracy: 0.9626 Epoch 149/150 60/60 - 1s - loss: 0.1672 - accuracy: 0.9567 - val_loss: 0.1652 - val_accuracy: 0.9578 Epoch 150/150 60/60 - 1s - loss: 0.1635 - accuracy: 0.9595 - val_loss: 0.1473 - val_accuracy: 0.9668
plt.figure(figsize = (30, 30))
plt.subplot(2,1,2)
plt.plot(hist.history['loss'])
plt.plot(hist.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper right')
<matplotlib.legend.Legend at 0x1da0276a9a0>
plt.figure(figsize = (30, 30))
plt.subplot(2,1,2)
plt.plot(hist.history['accuracy'])
plt.plot(hist.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'val'], loc='upper right')
<matplotlib.legend.Legend at 0x1da028e37c0>
